Reinforcement learning 34 Make continuous Agent videos

It is intended for junior high school students to university students who are new to AI. It is assumed that you have completed Reinforcement Learning 28.

In the middle of reinforcement learning, for example, save the agent every 10,000 steps, I would like to play it one by one continuously. I wonder if this makes it easier to see the growth process of learning. You can see it on Youtube, etc., and it's getting better and better.

If you do reinforcement learning with chokozainerRL, a folder will be created like this. To read this in order, do as follows.

import os
import re
testdir='mydrive/OpenAI/CartPole/result_dqn_choko'
files = os.listdir(testdir)
files_dir = [f for f in files if os.path.isdir(os.path.join(testdir, f))]
agentList=[]
for f in files_dir:
  if re.search('_',f):
    f2=f.split('_')
    agentList.append([int(f2[0]),f])
agentList.sort()

I will do it with Google Drive mounted. If you sort it as it is, it will be sorted as a character string, so the order will be strange. So, I will take out only the front number and sort it.

Assuming that env and agent are defined, do as follows.

from matplotlib import animation
import matplotlib.pyplot as plt

frames = []
for item in agentList:
  agent.load(testdir+'/'+item[1])
  obs = env.reset()
  done = False
  R = 0
  t = 0
  while not done and t < 200:
    frames.append(env.render(mode = 'rgb_array'))
    action = agent.act(obs)
    obs, r, done, _ = env.step(action)
    R += r
    t += 1
  print('test episode:', item[1], 'R:', R)
  agent.stop_episode()
env.close()
from IPython.display import HTML
plt.figure(figsize=(frames[0].shape[1]/72.0, frames[0].shape[0]/72.0),dpi=72)
patch = plt.imshow(frames[0])
plt.axis('off') 
def animate(i):
  patch.set_data(frames[i])
anim = animation.FuncAnimation(plt.gcf(), animate, frames=len(frames),interval=50)
anim.save(testdir+"grouth.mp4")
HTML(anim.to_jshtml())