Breakout with Tensorflow

Overview

Incorporating a machine learning model into the game is fun because you can see the results graphically. Therefore, create a model with Tensorflow and incorporate it into the game. Specifically, Tensorflow creates a model using human play data and breaks blocks. Don't worry about sophisticated models, make ** neat ** things ** neat **. The basic model creation in Tensorflow is in the following article Understand Tensorflow properly and create a model (Introduction to Tensorflow) The breakout to be used is the one linked below Breakout

Motivation

I often see articles that let machine learning models play a lot, but I'd like to try it out without spending a lot of time. The purpose is to move it so that you can feel it move without worrying about accuracy and beauty.

Premise

The outline of the breakout used this time is as follows.

Move the paddle to play the ball
Determine the coordinates of the paddle with the x coordinate of the mouse

In other words, when the y-coordinate of the ball and the y-coordinate of the paddle match, it is possible to play the ball as long as the x-coordinates of each match. In that sense, creating an auto breakout is fairly gentle.

Data creation

Create data from actual play. This time, the purpose is to proceed to ** Kito **, so I will not consider the set of explanatory variables and explained variables in detail. First of all, when actually playing, write out the coordinates of the paddle and the coordinates of the ball according to the update timing of the screen, the x coordinate and y coordinate of the ball that can be placed in each frame are the explanatory variables, and the position of the paddle is the explained variable. Create the data to be used. To do this, write the process of writing (saving) the coordinates in the move function of the Ball class and the update function of the Paddle class.

Creating a model

`model.py`


#Library import
import tensorflow as tf
import pandas as pd

#Data read
data_x = pd.read_csv('/somewhere/ball_movement_x.csv', header=None)
data_y = pd.read_csv('/somewhere/ball_movement_y.csv', header=None)
movement = pd.read_csv('/somewhere/mouse_movement.csv', header=None)

data = pd.DataFrame({'x_data': data_x.ix[:, 0],
                     'y_data': data_y.ix[:, 0]
                     })
teacher = pd.DataFrame({'teacher':movement.ix[:, 0]})

#Modeling
#Placeholder settings
X = tf.placeholder(tf.float32, shape = [None, 2])
Y = tf.placeholder(tf.float32, shape = [None, 1])

#Variable setting
W = tf.Variable(tf.zeros([2,1]))

B = tf.Variable(tf.zeros([1]))

y_hypo = tf.matmul(X, W) + B

#Loss function
cost = tf.reduce_mean(tf.square(y_hypo - Y))

#Gradient descent
train_step = tf.train.GradientDescentOptimizer(0.000001).minimize(cost)


with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    for i in range(1000):
        sess.run(train_step, feed_dict={X:data, Y:teacher})
    parameter_W = sess.run(W)
    parameter_B = sess.run(B)

Look in order

`model.py`


#Modeling
#Placeholder settings
X = tf.placeholder(tf.float32, shape = [None, 2])
Y = tf.placeholder(tf.float32, shape = [None, 1])

#Variable setting
W = tf.Variable(tf.zeros([2,1]))
B = tf.Variable(tf.zeros([1]))

y_hypo = tf.matmul(X, W) + B

In the learning phase, actual data is assigned to placeholders, and variables are determined as parameters during learning. y_hypo is a model for predicting paddle coordinates based on data and parameters.

`optimize.py`


#Loss function
cost = tf.reduce_mean(tf.square(y_hypo - Y))

#Gradient descent
train_step = tf.train.GradientDescentOptimizer(0.000001).minimize(cost)


with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    for i in range(1000):
        sess.run(train_step, feed_dict={X:data, Y:teacher})
    parameter_W = sess.run(W)
    parameter_B = sess.run(B)

Actually learn and search for parameters. parameter_W and parameter_B are the obtained parameters, X is the input data (x and y coordinates of the ball), and the value of tf.matmul (X, parameter_W) + parameter_B is the predicted coordinate of the paddle.

Run

Get the coordinates of the ball in the while of the main function, put it in the model above, and use the predicted value as the coordinates of the paddle. It is very heavy because the prediction is made every time the information is updated, but for the time being, it works as follows.

ezgif.com-resize.gif

Impressions

The purpose of this time was to incorporate the model into the game and make it work, but it worked. In reality, we will proceed while paying attention to what kind of data will be used and how to make the movement more sophisticated. For example, in the case of the breakout this time, when the blocks disappeared to a certain point, the movement of the paddle and the ball was fixed, and it came to go back and forth in the space without blocks all the time. In addition, it was inefficient to make predictions every time the frame was updated, and the ball moved very slowly.