I tried to make Othello AI with tensorflow without understanding the theory of machine learning ~ Battle Edition ~

Series table of contents

-I tried to make Othello AI with tensorflow without understanding the theory of machine learning-Introduction- -I tried to make Othello AI with tensorflow without understanding the theory of machine learning-Implementation- ――I tried to make Othello AI with tensorflow without understanding the theory of machine learning-Iza Battle- -I tried to make Othello AI after trying to understand the theory of machine learning ~ Restart! ~ -I tried to make Othello AI after trying to understand the theory of machine learning-What is this Alpha Zero edition- -I tried to make a neutral network with Excel to understand the theory of machine learning ~ Image recognition mnist edition ~

Continuation of Last time ... In this field, as an outsider, I didn't study "theory of machine learning" at all. I would like to make an AI for Othello.

Preparation

The opponent is here made by a friend ・ PHP Reversi

First, open the opponent's site and player white (select the second move) 対戦1.png

Start the prepared AI

python


$ python FightWithAI.py

------------- GAME START ---------------
***user turn ○***
  0  1  2  3  4  5  6  7
  8  9 10 11 12 13 14 15
 16 17 18 19 20 21 22 23
 24 25 26 ○ ● 29 30 31
 32 33 34 ● ○ 37 38 39
 40 41 42 43 44 45 46 47
 48 49 50 51 52 53 54 55
 56 57 58 59 60 61 62 63
Please enter the number
[44, 37, 26, 19]

>>>

I will enter the AI instructions.

Battle

First move (opponent) No. 19

j1.png

Gote (AI) No. 18

python


***AI turn ●***
  0  1  2  3  4  5  6  7
  8  9 10 11 12 13 14 15
 16 17 18 ● 20 21 22 23
 24 25 26 ● ● 29 30 31
 32 33 34 ● ○ 37 38 39
 40 41 42 43 44 45 46 47
 48 49 50 51 52 53 54 55
 56 57 58 59 60 61 62 63
Please enter the number
[44, 37, 26, 19]

>>> 18

First move (opponent) No. 26

j1.png

Gote (AI) No. 34

j2.png

First move (opponent) No. 42

j1.png

Gote (AI) No. 44

j2.png

First move (opponent) No. 43

j1.png

Gote (AI) No. 12

j2.png

First move (opponent) No. 29

j1.png

Gote (AI) No. 37

j2.png

First move (opponent) No. 21

j1.png

Gote (AI) No. 22

j2.png

First move (opponent) No. 23

j1.png

Gote (AI) No. 15

j2.png

First move (opponent) No. 7

j1.png

Gote (AI) No. 41

j2.png

First move (opponent) No. 3

j1.png

Gote (AI) No. 14

j2.png

First move (opponent) No. 5

j1.png

Gote (AI) No. 6

j2.png

First move (opponent) No. 20

j1.png

Gote (AI) No. 4

j2.png

First move (opponent) No. 45

j1.png

Gote (AI) No. 46

j2.png

First move (opponent) No. 11

j1.png

Gote (AI) No. 2

j2.png

First move (opponent) No. 1

j1.png

Gote (AI) No. 13

j2.png

First move (opponent) No. 17

j1.png

Gote (AI) No. 31

j2.png

First move (opponent) No. 39

j1.png

Gote (AI) No. 16

j2.png

First move (opponent) No. 25

j1.png

Gote (AI) No. 24

j2.png

First move (opponent) No. 32

j1.png

Gote (AI) No. 33

j2.png

First move (opponent) No. 8

j1.png

Gote (AI) No. 9

j2.png

First move (opponent) No. 0

j1.png

Gote (AI) No. 10

j2.png

First move (opponent) No. 40

j1.png

Gote (AI) pass

j2.png

First move (opponent) No. 47

j1.png

Gote (AI) No. 49

j2.png

First move (opponent) No. 56

j1.png

Gote (AI) No. 53

j2.png

First move (opponent) No. 48

j1.png

Gote (AI) No. 57

j2.png

First move (opponent) No. 58

j1.png

Gote (AI) No. 55

j2.png

First move (opponent) No. 63

j1.png

Gote (AI) No. 52

j2.png

First move (opponent) No. 62

j1.png

Gote (AI) No. 61

j2.png

First move (opponent) No. 54

j1.png

Gote (AI) No. 38

j2.png

First move (opponent) 59th

j1.png

Gote (AI) No. 30

j2.png

First move (opponent) No. 60

j1.png

Gote (AI) No. 51

j2.png

First move (opponent) No. 50

j1.png

result

Eh, it ’s a complete defeat orz Or rather, I spent a month making the weakest AI that lets me take all four corners.

f6a691dd-7b92-826a-36ea-fddc08fa9fc4.png

Even at the end of the training, Q_MAX was 0.023, which was too small, so I thought it was strange. Normally, at the end of training, the log should look like this and Q_MAX should be close to 1.

python


player:1 | pos:32 | LOSS: 0.0014 | Q_MAX: 0.9423
player:2 | pos:15 | LOSS: 0.0020 | Q_MAX: 0.9335
…

Entertainment AI is ready

Revenge

Next time Revenge by changing from tensorflow to chainer.

Apparently optimizers.RMSpropGraves, which cannot be used with tensorflow, is working I've been learning well so far. (At the time of writing, it's not strong yet)

If you are familiar with this, please let us know. m(_ _)m

Recommended Posts

I tried to make Othello AI with tensorflow without understanding the theory of machine learning ~ Battle Edition ~
I tried to make Othello AI with tensorflow without understanding the theory of machine learning ~ Introduction ~
I tried to make Othello AI with tensorflow without understanding the theory of machine learning ~ Implementation ~
[Machine learning] I tried to summarize the theory of Adaboost
I tried to make something like a chatbot with the Seq2Seq model of TensorFlow
I tried to make Othello AI that I learned 7.2 million hands by deep learning with Chainer
I tried to predict the presence or absence of snow by machine learning.
I tried to move machine learning (ObjectDetection) with TouchDesigner
I tried to compress the image using machine learning
I tried to understand the learning function of neural networks carefully without using a machine learning library (first half).
I tried to make a real-time sound source separation mock with Python machine learning
I tried to create a reinforcement learning environment for Othello with Open AI gym
I tried to find the entropy of the image with python
I tried to implement ListNet of rank learning with Chainer
I tried to rewrite the WEB server of the normal Linux programming 1st edition with C ++ 14
I tried to compare the accuracy of machine learning models using kaggle as a theme.
I tried to verify the yin and yang classification of Hololive members by machine learning
I tried machine learning with liblinear
I tried to understand it carefully while implementing the algorithm Adaboost in machine learning (+ I deepened my understanding of array calculation)
I tried to solve the ant book beginner's edition with python
I tried to automate the watering of the planter with Raspberry Pi
I tried to make deep learning scalable with Spark × Keras × Docker
I tried to improve the efficiency of daily work with Python
I tried to make a mechanism of exclusive control with Go
I tried to implement Autoencoder with TensorFlow
I tried to transform the face image using sparse_image_warp of TensorFlow Addons
I tried calling the prediction API of the machine learning model from WordPress
I tried to make a traffic light-like with Raspberry Pi 4 (Python edition)
I tried to automatically extract the movements of PES players with software
(Machine learning) I tried to understand Bayesian linear regression carefully with implementation.
I tried to analyze the negativeness of Nono Morikubo. [Compare with Posipa]
A beginner of machine learning tried to predict Arima Kinen with python
I tried to streamline the standard role of new employees with Python
Memorandum of means when you want to make machine learning with 50 images
I tried to get the movie information of TMDb API with Python
I tried the common story of using Deep Learning to predict the Nikkei 225
I tried to predict the behavior of the new coronavirus with the SEIR model.
I refactored "I tried to make Othello AI when programming beginners studied python"
I tried to understand the learning function in the neural network carefully without using the machine learning library (second half).
I tried to make a thumbnail image of the best avoidance flag-chan! With RGB values ​​[Histogram] [Visualization]
I tried to predict next year with AI
I tried to organize the evaluation indexes used in machine learning (regression model)
I tried to save the data with discord
I tried to touch the API of ebay
I tried to correct the keystone of the image
Machine learning beginners tried to make a horse racing prediction model with python
I tried to easily visualize the tweets of JAWS DAYS 2017 with Python + ELK
I tried object detection with YOLO v3 (TensorFlow 2.1) on the GPU of windows!
I tried to predict the change in snowfall for 2 years by machine learning
I tried to process and transform the image and expand the data for machine learning
I want to do machine learning even without a server --Time Series Edition -
I didn't understand the Resize of TensorFlow so I tried to summarize it visually.
I'm an amateur on the 14th day of python, but I want to try machine learning with scikit-learn
The story of making soracom_exporter (I tried to monitor SORACOM Air with Prometheus)
I tried to make a ○ ✕ game using TensorFlow
I tried to create a model with the sample of Amazon SageMaker Autopilot
I tried to automatically send the literature of the new coronavirus to LINE with Python
I tried to predict the price of ETF
I tried to vectorize the lyrics of Hinatazaka46!
I tried to put out the frequent word ranking of LINE talk with Python
I tried to solve the virtual machine placement optimization problem (simple version) with blueqat