Breakout with Tensorflow

Overview

Incorporating a machine learning model into the game is fun because you can see the results graphically. Therefore, create a model with Tensorflow and incorporate it into the game. Specifically, Tensorflow creates a model using human play data and breaks blocks. Don't worry about sophisticated models, make ** neat ** things ** neat **. The basic model creation in Tensorflow is in the following article Understand Tensorflow properly and create a model (Introduction to Tensorflow) The breakout to be used is the one linked below Breakout

Motivation

I often see articles that let machine learning models play a lot, but I'd like to try it out without spending a lot of time. The purpose is to move it so that you can feel it move without worrying about accuracy and beauty.

Premise

The outline of the breakout used this time is as follows.

In other words, when the y-coordinate of the ball and the y-coordinate of the paddle match, it is possible to play the ball as long as the x-coordinates of each match. In that sense, creating an auto breakout is fairly gentle.

Data creation

Create data from actual play. This time, the purpose is to proceed to ** Kito **, so I will not consider the set of explanatory variables and explained variables in detail. First of all, when actually playing, write out the coordinates of the paddle and the coordinates of the ball according to the update timing of the screen, the x coordinate and y coordinate of the ball that can be placed in each frame are the explanatory variables, and the position of the paddle is the explained variable. Create the data to be used. To do this, write the process of writing (saving) the coordinates in the move function of the Ball class and the update function of the Paddle class.

Creating a model

model.py


#Library import
import tensorflow as tf
import pandas as pd

#Data read
data_x = pd.read_csv('/somewhere/ball_movement_x.csv', header=None)
data_y = pd.read_csv('/somewhere/ball_movement_y.csv', header=None)
movement = pd.read_csv('/somewhere/mouse_movement.csv', header=None)

data = pd.DataFrame({'x_data': data_x.ix[:, 0],
                     'y_data': data_y.ix[:, 0]
                     })
teacher = pd.DataFrame({'teacher':movement.ix[:, 0]})

#Modeling
#Placeholder settings
X = tf.placeholder(tf.float32, shape = [None, 2])
Y = tf.placeholder(tf.float32, shape = [None, 1])

#Variable setting
W = tf.Variable(tf.zeros([2,1]))

B = tf.Variable(tf.zeros([1]))

y_hypo = tf.matmul(X, W) + B

#Loss function
cost = tf.reduce_mean(tf.square(y_hypo - Y))

#Gradient descent
train_step = tf.train.GradientDescentOptimizer(0.000001).minimize(cost)


with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    for i in range(1000):
        sess.run(train_step, feed_dict={X:data, Y:teacher})
    parameter_W = sess.run(W)
    parameter_B = sess.run(B)

Look in order

model.py


#Modeling
#Placeholder settings
X = tf.placeholder(tf.float32, shape = [None, 2])
Y = tf.placeholder(tf.float32, shape = [None, 1])

#Variable setting
W = tf.Variable(tf.zeros([2,1]))
B = tf.Variable(tf.zeros([1]))

y_hypo = tf.matmul(X, W) + B

In the learning phase, actual data is assigned to placeholders, and variables are determined as parameters during learning. y_hypo is a model for predicting paddle coordinates based on data and parameters.

optimize.py


#Loss function
cost = tf.reduce_mean(tf.square(y_hypo - Y))

#Gradient descent
train_step = tf.train.GradientDescentOptimizer(0.000001).minimize(cost)


with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    for i in range(1000):
        sess.run(train_step, feed_dict={X:data, Y:teacher})
    parameter_W = sess.run(W)
    parameter_B = sess.run(B)

Actually learn and search for parameters. parameter_W and parameter_B are the obtained parameters, X is the input data (x and y coordinates of the ball), and the value of tf.matmul (X, parameter_W) + parameter_B is the predicted coordinate of the paddle.

Run

Get the coordinates of the ball in the while of the main function, put it in the model above, and use the predicted value as the coordinates of the paddle. It is very heavy because the prediction is made every time the information is updated, but for the time being, it works as follows.

ezgif.com-resize.gif

Impressions

The purpose of this time was to incorporate the model into the game and make it work, but it worked. In reality, we will proceed while paying attention to what kind of data will be used and how to make the movement more sophisticated. For example, in the case of the breakout this time, when the blocks disappeared to a certain point, the movement of the paddle and the ball was fixed, and it came to go back and forth in the space without blocks all the time. In addition, it was inefficient to make predictions every time the frame was updated, and the ball moved very slowly.

Recommended Posts

Breakout with Tensorflow
Zundokokiyoshi with TensorFlow
Reading data with TensorFlow
Kyotei forecast with TensorFlow
Try regression with TensorFlow
Translate Getting Started With TensorFlow
Try deep learning with TensorFlow
Approximate sin function with TensorFlow
Jetson Nano JETPACK 44.1 (2020/10/21) with Tensorflow
Easy image classification with TensorFlow
Stock price forecast with tensorflow
Try TensorFlow MNIST with RNN
Ensure reproducibility with tf.keras in Tensorflow 2.3
TensorFlow 2.2 can't be installed with Python 3.8!
MNIST (DCNN) with Keras (TensorFlow backend)
Customize Model / Layer / Metric with TensorFlow
Classify "Wine" with TensorFlow MLP code
Precautions when installing tensorflow with anaconda
Try deep learning with TensorFlow Part 2
Let's make a breakout with wxPython
[TensorFlow] [Keras] Neural network construction with Keras
Use Tensorflow 2.1.0 with Anaconda on Windows 10!
Try data parallelism with Distributed TensorFlow
Zura predicting today's temperature with TensorFlow
Intellisense doesn't work with tensorflow2.0 + VScode
Achieve pytorch reflection padding with Tensorflow
I tried to implement Autoencoder with TensorFlow
Learn data distributed with TensorFlow Y = 2X
I tried to visualize AutoEncoder with TensorFlow
Put TensorFlow in P2 instance with pip3
Build a Tensorflow environment with Raspberry Pi [2020]
Compare raw TensorFlow with tf.contrib.learn and Keras
Nowadays, implement DQN (complete version) with Tensorflow
Numerical calculation of differential equations with TensorFlow 2.0
Stock Price Forecast with TensorFlow (LSTM) ~ Stock Forecast Part 1 ~
tensorflow mnist_deep.py
TensorFlow tutorial tutorial
Try TensorFlow RNN with a basic model
Video frame prediction using convolution LSTM with TensorFlow
Shuffle hundreds of thousands of images evenly with tensorflow.
Try Tensorflow with a GPU instance on AWS
Memory Leak occurred after learning DQN with tensorflow == 2.0.0
Load the TensorFlow model file .pb with readNetFromTensorflow ().
Calculate the angle between n-dimensional vectors with TensorFlow