From Unity ML-Agents release_6 Install to environmental testing on Win10.

Introduction

Since a new major version has been released since I used Unity ML-Agents last time, I will prepare the execution environment for the time being.

Reference: Unity Ml-Agents Installation

If you have any problems, see the official. Read English.

Execution environment

ML-Agents uses the latest Release 6 which is not currently unstable.

Also note that Release 6 must use Unity 2018.4 or later, and Python 3.6.1 or later.

First, create a virtual environment

Create a virtual environment in Anaconda so that it can be separated from the original environment.

conda create -n ml-agents

If you want to specify the version of python (assuming version 3.6),

conda create -n ml-agents pyhton=3.6

After creating the virtual environment, activate it.

conda activate ml-agents

This completes the virtual environment creation.

ML-Agents Release 6 setup

It is assumed that you have installed Unity and Python.

First, clone the Toolkit Repository from Github.

git clone --branch release_6 https://github.com/Unity-Technologies/ml-agents.git

--branch release_6 is a tag that specifies release 6, otherwise you will clone the repository of the master branch. The master branch is up-to-date, but an unstable version.

Install the com.unity.ml-agents package in Unity

There is a folder com.unity.ml-agents in the repository you cloned earlier, so install it in Unity.

1. Open Unity and open Window-> Package Manager
2. Click the + button in the Package Manager window
3. Select Add package from disk ...
4. Open the com.unity.ml-agents folder
5. Select the package.json file

Install the Python mlagents package

pip3 install mlagents

Try the Getting Started Guide

Reference: Getting Started Guide

There is an item in this guide to understand Environment, but if you want to create your own environment, you should read it carefully.

Since this is a trial, I will use the 3D Ball pre-trained model first. There are many other samples, so let's try them out.

Try the pre-trained model

1. Open 3DBall from Project / Assets / ML-Agents / Examples / 3DBall / Scenes
2. Open Assets / ML-Agents / Examples / 3DBall / Prefabs from the Project window, expand 3D Ball and click Agent prefab

Note: The 3D Ball platform is built using the 3D Ball prefab. So instead of updating all 12 platforms, you can change it by updating the 3D Ball prefab.

3. From the Project window, drag the 3DBall of Assets / ML-Agents / Examples / 3DBall / TFModels to the Model of Behavior Parameters of the Agent in the Inspector window. (Although it has already been set, you can check the learning result by saving each learning result as NNModel and setting it in Model.)
4. Confirm that the Model of Behavior Parameter of Agent of each 3D Ball of Scene is 3D Ball (NN Model).
5. Set the Inference Device used in this Model to the CPU (GPU if using GPU)
6. Press the Play button in Unity Editor and make sure 3D Ball is running

Learn a new model with reinforcement learning

1. Open terminal window

2. Go to the cloned ml-agents repository

3. Execute the following command

mlagents-learn config/ppo/3DBall.yaml --run-id=first3DBallRun ```

4. When the message "Start training by pressing the Play button in the Unity Editor" appears, press the Unity Play button to start learning.
5. The learning result is stored in ml-agents / results, so drag first3DBallRun to the Project window and copy it.
6. From the Project window, drag 3D Ball from first3DBallRun / run_logs to Model of Behavior Parameter of 3D Ball prefab.
7. Press the Play button in Unity to check the learning result

Check the progress of training

Check the result you learned earlier on the tensorboard

tensorboard --logdir results

If you execute the above command, you can check it at http: // localhost: 6006 in your browser.

Summary

This completes the process from installation to testing. Please let us know if you have any mistakes or shortcomings.

Also, since it was written for the purpose of confirming what I did, I think there are some points that are difficult to understand. Basically, you should read the official documentation and run it yourself.