Create a machine learning environment from scratch with Winsows 10

Introduction

Do you guys do it?

** Machine learning **

For the past few years, he has been touted as the third AI boom. Artificial intelligence, machine learning, neural networks, deep learning I've been studying silently lately, but ... it's kind of difficult ...

This time, let's set aside that area and create a machine learning environment quickly. It's surprisingly easy to make. (I think there are many reference articles, but they also serve as personal notes.)

If you're okay with Google Colaboratory Here

PC environment

・ ** OS **: Windows 10 Pro ・ ** GPU **: GeForce 940MX ・ ** CPU **: Core i5-7200U ・ ** Memory **: 16GB

Installation of nVIDIA CUDA Toolkit and cuDNN

First, let's install CUDA Toolkit and cuDNN by referring to this article. This is because it is required when using TensorFlow (GPU version).

As a caveat, if you download CUDA with the link in the article, it will be the latest 10.1, and you have to set a symbolic link when using it with TensorFlow, so [here](https: // developer. It is easier to download and install the 10.0 archive from nvidia.com/cuda-10.0-download-archive).

Installation of Anaconda

The Python environment uses ʻAnaconda`.

As a matter of fact, I once created a machine learning environment even in a plain Python 3.7 environment. And I thought I would write an article like " Creating a machine learning environment on Windows 10 without Anaconda ".

However, while researching, I found some articles that the environment installed with conda is faster, so "Well, then recreate the environment with Anaconnda?" With that in mind, I built a Python environment with Anaconda.

So, first download and install the installer from Anaconda Official.

halfway Add Anaconda to my PATH environment variavle If you check, I think that the Anaconda path will be included in the environment variable.

After installation, at the command prompt, PowerShell, etc.

conda python pip Let's execute the above three and check if the path is in place.

Create virtual environment

Anaconda makes it easy to create a Python environment. If you want to know more about the advantages, we will leave it to Google teacher this time.

The command to create a virtual environment is as follows

> conda create --name myenv python=3.7

The part of myenv is the name of the virtual environment, so please name it whatever you like. You can create a bang-bang environment with just this.

Switching the environment

> conda activate myenv

You can switch with.

> activate myenv

However, it seems to be an old version of writing, At first glance it looks like it switched without any errors, The environment has not been switched even if I check it with ʻinfo` described later.

To see which environment you are currently active in

> conda info -e

You can check it as follows.

# conda environments:
#
base                     C:\Users\<username>\Anaconda3
myenv                 *  C:\Users\<username>\Anaconda3\envs\myenv

The current environment is marked with "*".

Installation of packages that may be needed

You may not need it, as it says it seems necessary. I want to try various machine learning! It may be necessary in that case.

conda install pandas
conda install scikit-learn
conda install scikit-image

OpenAI Gym environment installation

pip install gym
conda install -c conda-forge jsanimation
conda install pyglet
conda install -c conda-forge ffmpeg
pip install --no-index -f https://github.com/Kojoley/atari-py/releases atari_py

There is a story that mixing conda install and pip install will break the environment, so I'd like to avoid using pip install if possible, but if Anaconda doesn't have a package, it's unavoidable ...

Keras and TensorFlow environment installation

If you have TensorFlow, it's included in Keras, so you don't need it, right? I think there is an opinion, but just in case, it may be mixed in the sample.

conda install -c conda-forge keras
conda install pydotplus
conda install tensorflow
conda install tensorflow-gpu

Complete!

With the above, the environment for machine learning has been set up to some extent.

Final `conda list`
# Name                    Version                   Build  Channel
_tflow_select             2.1.0                       gpu
absl-py                   0.8.1                    py37_0
astor                     0.8.0                    py37_0
astroid                   2.3.2                    py37_0
atari-py                  1.2.1                    pypi_0    pypi
blas                      1.0                         mkl
ca-certificates           2019.10.16                    0
certifi                   2019.9.11                py37_0
cloudpickle               1.2.2                      py_0
colorama                  0.4.1                    py37_0
cudatoolkit               10.0.130                      0
cudnn                     7.6.4                cuda10.0_0
cycler                    0.10.0                   py37_0
cytoolz                   0.10.0           py37he774522_0
dask-core                 2.6.0                      py_0
decorator                 4.4.1                      py_0
ffmpeg                    4.2                  h6538335_0    conda-forge
freetype                  2.9.1                ha9979f8_1
future                    0.18.2                   pypi_0    pypi
gast                      0.2.2                    py37_0
google-pasta              0.1.7                      py_0
grpcio                    1.16.1           py37h351948d_1
gym                       0.15.4                   pypi_0    pypi
h5py                      2.9.0            py37h5e291fa_0
hdf5                      1.10.4               h7ebc959_0
icc_rt                    2019.0.0             h0cc432a_1
icu                       58.2                 ha66f8fd_1
imageio                   2.6.1                    py37_0
intel-openmp              2019.4                      245
isort                     4.3.21                   py37_0
joblib                    0.14.0                     py_0
jpeg                      9b                   hb83a4c4_2
jsanimation               0.1                        py_1    conda-forge
keras                     2.3.1            py37h21ff451_0    conda-forge
keras-applications        1.0.8                      py_0
keras-preprocessing       1.1.0                      py_1
kiwisolver                1.1.0            py37ha925a31_0
lazy-object-proxy         1.4.3            py37he774522_0
libgpuarray               0.7.6             hfa6e2cd_1003    conda-forge
libpng                    1.6.37               h2a8f88b_0
libprotobuf               3.9.2                h7bd577a_0
libtiff                   4.1.0                h56a325e_0
mako                      1.1.0                      py_0    conda-forge
markdown                  3.1.1                    py37_0
markupsafe                1.1.1            py37hfa6e2cd_0    conda-forge
matplotlib                3.1.1            py37hc8f65d3_0
mccabe                    0.6.1                    py37_1
mkl                       2019.4                      245
mkl-service               2.3.0            py37hb782905_0
mkl_fft                   1.0.15           py37h14836fe_0
mkl_random                1.1.0            py37h675688f_0
networkx                  2.4                        py_0
numpy                     1.17.3           py37h4ceb530_0
numpy-base                1.17.3           py37hc3f5095_0
olefile                   0.46                     py37_0
opencv-python             4.1.1.26                 pypi_0    pypi
openssl                   1.1.1d               he774522_3
opt_einsum                3.1.0                      py_0
pandas                    0.25.2           py37ha925a31_0
pillow                    6.2.1            py37hdc69c19_0
pip                       19.3.1                   py37_0
protobuf                  3.9.2            py37h33f27b4_0
pydotplus                 2.0.2                    py37_1
pyglet                    1.3.2                    pypi_0    pypi
pygpu                     0.7.6           py37hc8d92b1_1000    conda-forge
pylint                    2.4.3                    py37_0
pyparsing                 2.4.2                      py_0
pyqt                      5.9.2            py37h6538335_2
pyreadline                2.1                      py37_1
python                    3.7.5                h8c8aaf0_0
python-dateutil           2.8.1                      py_0
pytz                      2019.3                     py_0
pywavelets                1.1.1            py37he774522_0
pyyaml                    5.1.2            py37hfa6e2cd_0    conda-forge
qt                        5.9.7            vc14h73c81de_0
scikit-image              0.15.0           py37ha925a31_0
scikit-learn              0.21.3           py37h6288b17_0
scipy                     1.3.1            py37h29ff71c_0
setuptools                41.6.0                   py37_0
sip                       4.19.8           py37h6538335_0
six                       1.12.0                   py37_0
sqlite                    3.30.1               he774522_0
tensorboard               2.0.0              pyhb230dea_0
tensorflow                2.0.0           gpu_py37h57d29ca_0
tensorflow-base           2.0.0           gpu_py37h390e234_0
tensorflow-estimator      2.0.0              pyh2649769_0
tensorflow-gpu            2.0.0                h0d30ee6_0
termcolor                 1.1.0                    py37_1
theano                    1.0.4           py37h6538335_1000    conda-forge
tk                        8.6.8                hfa6e2cd_0
toolz                     0.10.0                     py_0
tornado                   6.0.3            py37he774522_0
tqdm                      4.36.1                     py_0
vc                        14.1                 h0510ff6_4
vs2015_runtime            14.16.27012          hf0eaf9b_0
vs2015_win-64             14.0.25420          h55c1224_11
webencodings              0.5.1                    py37_1
werkzeug                  0.16.0                     py_0
wheel                     0.33.6                   py37_0
wincertstore              0.2                      py37_0
wrapt                     1.11.2           py37he774522_0
xz                        5.2.4                h2fa13f4_4
yaml                      0.1.7             hfa6e2cd_1001    conda-forge
zlib                      1.2.11               h62dcd97_3
zstd                      1.3.7                h508b16e_0

Summary

I installed it steadily, and now I have various things in one virtual environment. I think it's okay to create a separate virtual environment and divide it according to the use.

After that, I searched for a sample and tried to move it, I think you will be able to try and error yourself and try various things!

・ ・ ・ I'd like to say, but the sample of TensorFlow 2.0.0 hasn't rolled yet ... There are many samples of past versions, but there are many things that have changed in 2.0.0, and many things do not work as they are.

TensorFlow Official is also available for tutorials on Google Colaboratory. (It's natural because it's a library developed by Google)

So, it seems that you still need to have enough knowledge to understand the old version of the sample and modify it ... \ _ (┐ "ε :) _

I'm still studying about machine learning, but if I can do something interesting in the future, As long as the memory of my brain catches up, I want to send it! ~~ (already bread bread) ~~

Recommended Posts

Create a machine learning environment from scratch with Winsows 10
Create an environment for "Deep Learning from scratch" with Docker
Create a game UI from scratch with pygame2!
Machine learning starting from scratch (machine learning learned with Kaggle)
Build a Python machine learning environment with a container
Build a machine learning environment
Build a machine learning application development environment with Python
Build a machine learning scikit-learn environment with VirtualBox and Ubuntu
Create a machine learning app with ABEJA Platform + LINE Bot
Create a python machine learning model relearning mechanism with mlflow
How to quickly create a machine learning environment using Jupyter Notebook with UbuntuServer 16.04 LTS
Until you create a machine learning environment with Python on Windows 7 and run it
How to quickly create a machine learning environment using Jupyter Notebook with UbuntuServer 16.04 LTS with anaconda
Building a Windows 7 environment for getting started with machine learning with Python
[Python] Create a virtual environment with Anaconda
A story about machine learning with Kyasuket
Create a virtual environment with Python_Mac version
[Windows 10] "Deep Learning from scratch" environment construction
Create an arbitrary machine learning environment with GCP + Docker + Jupyter Lab
Creating a development environment for machine learning
Build AI / machine learning environment with Python
How to create a serverless machine learning API with AWS Lambda
How to quickly create a machine learning environment using Jupyter Notebook on macOS Sierra with anaconda
Realize environment construction for "Deep Learning from scratch" with docker and Vagrant
Build a "Deep learning from scratch" learning environment on Cloud9 (jupyter miniconda python3)
Building a machine learning environment with Tellus GPU server (Sakura high-power computing)
Create a virtual environment with conda in Python
Create a python3 build environment with Sublime Text3
Create a decision tree from 0 with Python (1. Overview)
Deep Learning from scratch
Create a Python environment
Prepare the environment for O'Reilly's book "Deep Learning from scratch" with apt-get (Debian 8)
Run a machine learning pipeline with Cloud Dataflow (Python)
Create a virtual environment with Anaconda installed via Pyenv
Non-information graduate student studied machine learning from scratch # 1: Perceptron
Create a python development environment with vagrant + ansible + fabric
Let's feel like a material researcher with machine learning
Study method for learning machine learning from scratch (March 2020 version)
Create a Japanese OCR environment with Anaconda (tesseract + pyocr)
Memo for building a machine learning environment using Python
Create a django environment with docker-compose (MariaDB + Nginx + uWSGI)
An introduction to machine learning from a simple perceptron
Build a machine learning environment natively on Windows 10 (x64)
How to set up a Google Colab environment with Coursera's advanced machine learning courses
Steps to quickly create a deep learning environment on Mac with TensorFlow and OpenCV
Machine learning learned with Pokemon
Deep Learning from scratch 1-3 chapters
Create an environment with virtualenv
Create a homepage with django
Machine learning with Python! Preparation
Machine learning Minesweeper with PyTorch
Machine learning environment construction macbook 2021
Create a heatmap with pyqtgraph
Beginning with Python machine learning
Create a directory with python
Try machine learning with Kaggle
Rebuilding an environment for machine learning with Miniconda (Windows version)
Build a python machine learning study environment on macOS sierra
Build a machine learning environment on mac (pyenv, deeplearning, opencv)
Try to build a deep learning / neural network with scratch
Machine learning with Jupyter Notebook in OCI Always Free environment (2019/12/17)