I tried playing a ○ ✕ game using TensorFlow

Introduction

-Last time, using the created learning model, I actually played against AI in the ○ ✕ game. ――The first move is fixed to AI, and it seems that they are randomly placed. ―― ○ ✕ You can win the game if you hit it in the middle later, so it seems that it is not an AI that will definitely win.

Source code

Save model

ticktacktoo.py


#Please add at the bottom.

# Remove old model
if os.path.exists("model"):
    shutil.rmtree("model")

# Save model for deployment on ML Engine
input_key = tf.placeholder(tf.int64, [None, ], name="key")
output_key = tf.identity(input_key)

input_signatures = {
    "key": tf.saved_model.utils.build_tensor_info(input_key),
    "squares": tf.saved_model.utils.build_tensor_info(squares_placeholder)
}

output_signatures = {
    "key": tf.saved_model.utils.build_tensor_info(output_key),
    "labels": tf.saved_model.utils.build_tensor_info(logits)
}

predict_signature_def = tf.saved_model.signature_def_utils.build_signature_def(
    input_signatures,
    output_signatures,
    tf.saved_model.signature_constants.PREDICT_METHOD_NAME
)

builder = tf.saved_model.builder.SavedModelBuilder(os.path.join("model"))

builder.add_meta_graph_and_variables(
    sess,
    [tf.saved_model.tag_constants.SERVING],
    signature_def_map={
        tf.saved_model.signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY: predict_signature_def
    },
    assets_collection=tf.get_collection(tf.GraphKeys.ASSET_FILEPATHS)
)

builder.save()

Creating a game board

game.py


#!/usr/local/bin/python
# -*- coding: utf-8 -*-

import numpy as np

class Setting:

    def __init__(self):

        self.board = None
        self.current_player = None
        self.result = None
        self.reset()

    def reset(self):

        self.board = np.zeros(9, dtype=np.int32)
        self.current_player = "x"

    def step(self, index):

        if self.board[index] != 0:
            print("Invalid move!!")
            return None, None, None, {"valid": False}
        elif self.current_player == "x":
            self.board[index] = 1
            self.current_player = "o"
        else:
            self.board[index] = -1
            self.current_player = "x"

        observation = np.array(self.board)
        done, info = self.check_game_result()
        reward = 0

        return observation, reward, done, info

    def render(self):

        markers = []

        for i in self.board:
            if i == 0:
                markers.append("_")
            elif i == 1:
                markers.append("x")
            else:
                markers.append("o")

        print("{} is thinking...".format(self.current_player))
        print("{0}\t{1}\t{2}".format(markers[0], markers[1], markers[2]))
        print("{0}\t{1}\t{2}".format(markers[3], markers[4], markers[5]))
        print("{0}\t{1}\t{2}\n".format(markers[6], markers[7], markers[8]))

    def check_game_result(self):

        x_win, o_win, is_full = False, False, False

        # Check rows and cols
        for i in range(3):
            row = self.board[(i * 3):(i * 3 + 3)]
            col = self.board[i::3]

            if np.sum(row) == 3 or np.sum(col) == 3:
                x_win = True

            if np.sum(row) == -3 or np.sum(col) == -3:
                o_win = True

        # Check diag
        if np.sum(self.board[[0, 4, 8]]) == 3 or np.sum(self.board[[2, 4, 6]]) == 3:
            x_win = True

        if np.sum(self.board[[0, 4, 8]]) == -3 or np.sum(self.board[[2, 4, 6]]) == -3:
            o_win = True

        if 0 not in self.board:
            is_full = True

        done = x_win or o_win or is_full
        info = {"x": x_win, "o": o_win, "full": is_full, "valid": True}

        return done, info

Creating a game launch

play.py


#!/usr/local/bin/python
# -*- coding: utf-8 -*-

import sys
import numpy as np
import tensorflow as tf
import game

if __name__ == '__main__':

    with tf.Graph().as_default() as graph:

        sess = tf.Session()

        meta_graph = tf.saved_model.loader.load(
            sess=sess,
            tags=[tf.saved_model.tag_constants.SERVING],
            export_dir='model'
        )

        model_signature = meta_graph.signature_def[tf.saved_model.signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY]

        input_signature = model_signature.inputs
        output_signature = model_signature.outputs

        # Get names of input and output tensors
        input_tensor_name = input_signature['squares'].name
        output_tensor_name = output_signature['labels'].name

        # Get input and output tensors
        squares = sess.graph.get_tensor_by_name(input_tensor_name)
        labels = sess.graph.get_tensor_by_name(output_tensor_name)

    env = game.Setting()
    observation = env.reset()

    done = False
    info = None

    rule = """
Input your move!
[0] top-left-square
[1] top-middle-square
[2] top-right-square
[3] middle-left-square
[4] middle-middle-square
[5] middle-right-square
[6] bottom-left-square
[7] bottom-middle-square
[8] bottom-right-square
"""

    print(rule)

    for _ in range(9):

        env.render()

        if done:
            if info["x"]:
                print("x win!")
            elif info["o"]:
                print("o win!")
            else:
                print("Draw!")
            break

        # Compute scores
        prob_x_win = -np.ones(9)
        prob_o_win = np.ones(9)

        # prob_draw = np.zeros(9)
        for i in range(9):
            if env.board[i] == 0:
                board_copy = np.array([env.board])
                board_copy[0][i] = 1

                prob = sess.run(labels, feed_dict={squares: board_copy})

                # print i, prob
                prob_x_win[i] = prob[0][0]
                prob_o_win[i] = prob[0][1]
                # prob_draw = prob[0][2]

        # Decide CPU's move
        if max(prob_x_win) >= 0.05:
            cpu_move = prob_x_win.argmax()
        else:
            cpu_move = prob_o_win.argmin()

        _, _, done, info = env.step(cpu_move)

        env.render()

        if done:
            if info["x"]:
                print("x win!")
            elif info["o"]:
                print("o win!")
            else:
                print("Draw!")
            break

        while True:
            sys.stdout.write("Input your move: ")
            player_move = input()
            _, _, done, info = env.step(player_move)

            if info["valid"]:
                break

Resave model

--Save again in SavedModel format. --Just re-execute the learning script again.

Re-execute

python ticktacktoo.py

After re-execution

The following files are created under the model directory.

saved_model.pb variables

Launch the game

--The following game board is not output. ――As mentioned above, the AI side randomly inputs the first move. --Players can type by entering numbers on the keyboard with reference to Input your move !.

2017-07-05 14:03:59.615507: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-07-05 14:03:59.615531: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-07-05 14:03:59.615536: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2017-07-05 14:03:59.615540: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.

Input your move!
[0] top-left-square
[1] top-middle-square
[2] top-right-square
[3] middle-left-square
[4] middle-middle-square
[5] middle-right-square
[6] bottom-left-square
[7] bottom-middle-square
[8] bottom-right-square

x is thinking...
_	_	_
_	_	_
_	_	_

o is thinking...
_	_	x
_	_	_
_	_	_

I tried playing

draw

x is thinking...
_    _    _
_    _    _
_    _    _

o is thinking...
_    _    x
_    _    _
_    _    _

Input your move: 4
x is thinking...
_    _    x
_    o    _
_    _    _

o is thinking...
x    _    x
_    o    _
_    _    _

Input your move: 1
x is thinking...
x    o    x
_    o    _
_    _    _

o is thinking...
x    o    x
_    o    _
_    x    _

Input your move: 5
x is thinking...
x    o    x
_    o    o
_    x    _

o is thinking...
x    o    x
x    o    o
_    x    _

Input your move: 6
x is thinking...
x    o    x
x    o    o
o    x    _

o is thinking...
x    o    x
x    o    o
o    x    x

Draw!

victory

x is thinking...
_    _    _
_    _    _
_    _    _

o is thinking...
x    _    _
_    _    _
_    _    _

Input your move: 4
x is thinking...
x    _    _
_    o    _
_    _    _

o is thinking...
x    _    x
_    o    _
_    _    _

Input your move: 1
x is thinking...
x    o    x
_    o    _
_    _    _

o is thinking...
x    o    x
_    o    _
x    _    _

Input your move: 3
x is thinking...
x    o    x
o    o    _
x    _    _

o is thinking...
x    o    x
o    o    x
x    _    _

Input your move: 7
x is thinking...
x    o    x
o    o    x
x    o    _

o win!

Summary

――It was the most suitable algorithm ever. (Maybe it's easy) ――Imagine that there are multiple boards and methods such as AlphaGo and Shogi, and I heard that it would not be an ordinary effort. ――I wonder if you haven't done it later. Per sentiment analysis? ――It may be time to buy a book and read it carefully.

All page links

-I made a ○ ✕ game using TensorFlow -I tried playing a game using TensorFlow

Recommended Posts

I tried playing a ○ ✕ game using TensorFlow
I tried to make a ○ ✕ game using TensorFlow
I tried playing a typing game in Python
I tried using magenta / TensorFlow
I tried hosting a TensorFlow deep learning model using TensorFlow Serving
I tried drawing a line using turtle
I tried to classify text using TensorFlow
I tried using pipenv, so a memo
I tried using parameterized
I tried using argparse
I tried using mimesis
I tried using anytree
〇✕ I made a game
I tried using aiomysql
I tried using Summpy
I tried using coturn
I tried using Pipenv
I tried using matplotlib
I tried using "Anvil".
I tried using Hubot
I tried using ESPCN
I tried using openpyxl
I tried using Ipython
I tried using PyCaret
I tried using cron
I tried using ngrok
I tried using face_recognition
I tried using Jupyter
I tried using PyCaret
I tried using Heapq
I tried using doctest
I tried running TensorFlow
I tried using folium
I tried using jinja2
I tried using folium
I tried using time-window
I tried using Pythonect, a dataflow programming language.
I tried reading a CSV file using Python
I tried using a database (sqlite3) with kivy
I made a Dir en gray face classifier using TensorFlow --⑬ Playing (final)
I tried to implement a card game of playing cards in Python
[I tried using Pythonista 3] Introduction
I tried using easydict (memo).
I tried face recognition using Face ++
I tried using Random Forest
I tried using BigQuery ML
I tried hosting a Pytorch sample model using TorchServe
I tried using Amazon Glacier
I tried using git inspector
[Python] I tried using OpenPose
[Python] I tried running a local server using flask
I tried drawing a pseudo fractal figure using Python
I tried reading data from a file using Node.js.
I made a school festival introduction game using Ren’py
I tried using Python (3) instead of a scientific calculator
PyTorch Learning Note 2 (I tried using a pre-trained model)
I tried using AWS Chalice
I tried to draw a configuration diagram using Diagrams
I tried using Slack emojinator
I tried refactoring the CNN model of TensorFlow using TF-Slim
I tried using Tensorboard, a visualization tool for machine learning