Building a Raspberry Pi for studying Python and machine learning (RaspberryPi4 & Buster version (RaspberryPi3 is also possible))

[Details]

-** 2020/02/19 We reviewed some contents and confirmed the operation with Raspberry Pi 3 model B. ** ** -** Added Node-RED environment construction. ** **

Finally, the Raspberry Pi 4 model B is now available in Japan. I rewrote the previously posted [Building Raspberry Pi for Python and machine learning study] to the 4th edition of Raspberry Pi so that I can use it immediately. Besides the hardware, I have updated the version of Raspbian to the latest version (Buster 20200213). Also, it has been confirmed that the environment can be built with the same procedure with ** Raspberry Pi 3 model B **.

As of February 19, 2020, there are the following problems.

--The latest version of TensorFlow that can be installed with pip3 is 1.14.0 (as of 02/19/2020) --ʻImport tensorflow` I get an error when executing (can I ignore it?) -~~ Edge TPU is not officially supported (← supported by unofficial procedure) ~~ --~~ The latest version 4.1.1.26 of OpenCV that can be installed with pip3 does not work (← works if 4.1.0.25) ~~ -USB camera cannot be used with OpenCV that can be installed with ~~ pip3. (← Supported by apt install 3.2.0) ~~ -~~ There is no screen because I don't have a Micro HDMI cable ^^; (← Headless setup) ~~

[What you can do with the Raspberry Pi made in this procedure]

--Basic learning of Python --Data visualization (pandas, matplotlib)

【System configuration】

--Raspberry Pi 3 or 4 Model B

[0. Preparation of OS image]

I'm assuming you're using Raspbian Buster 20200213. As of February 19, 2020, you can download it from the official website, but I will write a mirror site just in case. [[Raspbian Buster 20200213 Mirror Site (ftp.jaist.ac.jp)]](http://ftp.jaist.ac.jp/pub/raspberrypi/raspbian/images/raspbian-2020-02-14/2020-02 -13-raspbian-buster.zip)

[1. Write and start OS image]

1-1. Writing OS image

Write the OS image prepared above to the SD card. Please refer to the following article for the procedure for writing to the SD card.

-[Set up Raspberry Pi Headless]

1-2. Creating a file for starting ssh

The current version of Raspbian has the ssh service disabled by default. Create an empty file called ssh (without extension) in the" Boot "partition of the SD card so that it can boot automatically. If you are using windows, the extension is hidden by default, so please check if there is an extension.

The ssh file is read and deleted when you start the Raspberry Pi.

1-3. Creating a configuration file for Wifi connection

As with ssh, if you put a file for Wifi settings in the "Boot" partition, it will be automatically read and written to the appropriate location.

Specifically, create a file named wpa_supplicant.conf in the" Boot "partition and describe the settings. Please describe the following contents to connect to Wifi. The settings need to be changed according to the access point to be connected.

wpa_supplicant.conf


ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1
country=JP

network={
    ssid="<SSID name>"
    psk="<password>"
}

If you need to authenticate with the RADIUS server linked with Active Directory, please refer to the following article. [Connect from Raspeye to a wireless LAN that requires RADIUS authentication linked with AD (WPA2-EAP)]-Qiita

1-4. Start Raspberry Pi

When the above preparations are complete, remove the SD card from the PC, insert it into the Raspberry Pi body, and turn on the power.

[2. System Update & Network Settings]

2-1. System update

When the Raspberry Pi starts, connect with ssh or execute the following command in the terminal of the console to update the system. (It takes a few minutes)

System update


sudo apt update
sudo apt upgrade -y
sudo reboot

2-2. Disable ipv6 (optional)

It's not required, but it can get in the way in some cases, so disable it. This setting is not required for ipv6 operation.

ipv6 invalidation setting


sudo vi /etc/sysctl.conf

[Settings]

/etc/sysctl.conf


net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1

[Reflect settings]

Reflect settings


sudo sysctl -p

Or

sudo reboot

2-3. Fixed IP address (optional)

Fix the IP address if necessary. This setting is not required when using in a DHCP environment. Since this procedure assumes operation on a wireless LAN, the IP is fixedly set for the wireless LAN adapter or SSID. Specifically, add the following contents to /etc/dhcpcd.conf. Change <IP address>, <router address>, <DNS address>, and <your_ssid> according to your environment.

/etc/dhcpcd.conf


interface wlan0
inform <IP address>
static routers=<Router address>
static domain_name_servers=<DNS address>
noipv6

Or

SSID <your_ssid>
inform <IP address>
static routers=<Router address>
static domain_name_servers=<DNS address>
noipv6

When using in multiple wireless AP environments, it is convenient to fix with SSID.

2-4. Turn off the wireless LAN Power Management (optional)

This setting is not required when using in a wired LAN environment. By default, the wireless LAN Power Management function is turned on, but this function may cause Wifi communication to become unstable. Add a command to turn off Power Management to rc.local that is executed at startup.

rc.Editing local


sudo vi /etc/rc.local

rc.local


#Add the following before the exit statement and save
sudo iwconfig wlan0 power off

Reboot


sudo reboot

[3. Installation of required modules]

Install the modules required by the following operations. (Mainly used in OpenCV and Numpy)

Required module installation


sudo apt install -y libhdf5-dev libqtwebkit4 libqt4-test libatlas-base-dev libjasper-dev

[4. Installation related to Python3]

4-1. Installation of basic modules

Python3&pip3 installation


sudo apt install python3 python3-dev -y

This operation was not necessary because Buster was pre-installed.

4-1-1. Update of pip3 (optional)

** This step does not need to be performed **

If you follow the previous steps and update the pip3 command, it will throw an error and stop working when you run pip3.

Upgrade of pip3 [Failure]


#If you execute the following command, pip3 cannot be executed.
sudo pip3 install pip -U
pip3 -V

#If pip3 cannot be operated with the above command, recover with the following command
sudo python3 -m pip uninstall pip
sudo apt install python3-pip --reinstall

For the time being, it is possible to update the pip3 command by the following procedure, but it was not necessary in this procedure.

pip3 upgrade(option)


#Uninstall existing pip3 command
sudo apt remove python3-pip

#Install the latest pip3 command
wget https://bootstrap.pypa.io/get-pip.py
sudo python3 get-pip.py
pip3 -V

#Update setuptools
sudo pip3 install setuptools -U

4-2. Installation of various modules of Python3

Install various modules with the pip command. I have specified the versions of ~~ numpy and pandas, but since the latest binary package does not exist at the time of environment construction and the build starts and it takes time to install, the previous version is specified. Regarding TensorFlow, the latest one whose operation has been confirmed at the time of environment construction is specified. ~~ Each module specifies the latest version that can be package installed with Python 3.7 series as of February 19, 2020. Please change the version as needed.

Installation of various modules for Python3(2020/02/19 points)


# numpy
sudo pip3 install numpy==1.18.1

# scipy
sudo pip3 install scipy==1.4.1

# sklearn
sudo pip3 install scikit-learn==0.22.1

# matplotlib
sudo pip3 install matplotlib==3.1.3

# pandas
sudo pip3 install pandas==1.0.1

# seaborn
sudo pip3 install seaborn==0.10.0


# Tensorflow (2020/02/The latest is 1 as of 19.14.0)
sudo pip3 install tensorflow
#If you have updated pip3`--ignore-installed`Correspondence with options
#sudo pip3 install tensorflow --ignore-installed 

# keras (tf.Not needed if you use keras)
# sudo pip3 install keras

# flask
sudo pip3 install flask flask_cors -U

# OpenCV
sudo pip3 install opencv-python==4.1.1.26
#Change and reflect environment variables(← I have to do this`import cv2`I get an error at runtime)
echo "export LD_PRELOAD=/usr/lib/arm-linux-gnueabihf/libatomic.so.1" | tee -a ~/.bashrc
source ~/.bashrc

# cv2.If you can't use multiple cameras with VideoCapture, install apt for provisional support.(However 3.2.0)
# sudo pip3 uninstall opencv-python
# sudo apt install python3-opencv

In addition, it is the most that pip is downloading the package, probably because "https://www.piwheels.org/" that manages the package recently is not working well. In many cases, the session will be disconnected during the process and the installation will fail as a result. Re-execution often solves the problem, but large download sizes such as scipy and tensorflow are not very successful. In that case, copy the URL of the package being downloaded that is displayed when you execute pip install, and download the file to your local folder with the wget command. The wget command has a resume function even if the download fails, so it will automatically retry and somehow download the target file. Then run sudo pip3 install <downloaded file> to install.

If you have a mirror site, you can point the reference to pip there, but I didn't know at the time of writing this article, so if anyone knows it, I would appreciate it if you could let me know.

5. Installation of Edge TPU

This item is not mandatory. Only run if you are using the Edge TPU.

Installing the library for EdgeTPU


echo "deb https://packages.cloud.google.com/apt coral-edgetpu-stable main" | sudo tee /etc/apt/sources.list.d/coral-edgetpu.list

curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

sudo apt update

#normal version
sudo apt install libedgetpu1-std
#Or the maximum clock version
# sudo apt install libedgetpu1-max

TF Lite installation


sudo pip3 install https://dl.google.com/coral/python/tflite_runtime-2.1.0.post1-cp37-cp37m-linux_armv7l.whl

Edge_TPU_Python_API installation


sudo apt install python3-edgetpu

【6. Jupyter Lab】 Install Jupyter Lab, which will give you a powerful Notebook environment.

6-1. Basic installation

[Installation] Install with the following command.

JupyterLab installation


sudo pip3 install jupyterlab

【Setting】 After the installation is completed, create a configuration file with the following command, and then edit the created configuration file. In the procedure below, you can connect from any terminal and set to require a password when connecting.

Creating a configuration file


jupyter notebook --generate-config

[Edit configuration file]

Edit configuration file


vi ~/.jupyter/jupyter_notebook_config.py

[Settings]

jupyter_notebook_config.py


c.NotebookApp.ip = '*'
c.NotebookApp.open_browser = False
c.NotebookApp.password = 'sha1:<Hashed password>'

You can get the hash of the password with the following command.

Password hashing


python3 -c 'from notebook.auth import passwd;print(passwd())'
(After executing the above, enter the characters you want to hash)

6-2. Launching Jputer Lab and turning it into a service

Start Jupyter Lab with the following command.

Start Jupyter Lab


jupyter lab

For service conversion, first prepare the configuration file for the service.

File preparation for service


sudo vi /etc/systemd/system/jupyter.service

The contents of the file are as follows.

/etc/systemd/system/jupyter.service


[Unit]
Desctiption = Jupyter Lab
After = syslog.target

[Service]
Type = simple
WorkingDirectory = /home/pi
Restart = always
ExecStart = /usr/local/bin/jupyter lab
User = pi
Group = pi

[Install]
WantedBy = multi-user.target

When the file is ready, enable and start the service with the following command.

command


#Service activation
sudo systemctl enable jupyter.service 

#Start of the service
sudo systemctl start jupyter.service 

6-3. Connect from a browser

With Jupyter Lab running above, connect to the following URL from your browser. In the above procedure, it is set to connect from other PCs, so please try connecting to Jupyter Lab from your own PC.

http://<ip_address>:8888/lab

You will be asked for a password, so enter the password you set above.

image.png

【7. Node-RED】

7-1. Installing Node-RED

You can create an environment to execute Node-RED by executing the following command.

Node-RED installation


bash <(curl -sL https://raw.githubusercontent.com/node-red/linux-installers/master/deb/update-nodejs-and-nodered)

The following message will be displayed. Press y and press Enter. ** If the old node.js is installed, it will be replaced with the new one, so please do not perform this procedure if there is a problem. ** ** This procedure assumes installation from a clean state, so execute it as it is.

This script will remove versions of Node.js prior to version 7.x, and Node-RED and
if necessary replace them with Node.js 10.x LTS (dubnium) and the latest Node-RED from Npm.

It also moves any Node-RED nodes that are globally installed into your user
~/.node-red/node_modules directory, and adds them to your package.json, so that
you can manage them with the palette manager.

It also tries to run 'npm rebuild' to refresh any extra nodes you have installed
that may have a native binary component. While this normally works ok, you need
to check that it succeeds for your combination of installed nodes.

To do all this it runs commands as root - please satisfy yourself that this will
not damage your Pi, or otherwise compromise your configuration.
If in doubt please backup your SD card first.

Are you really sure you want to do this ? [y/N] ?

The following message will be displayed, so select y to proceed.

Would you like to install the Pi-specific nodes ? [y/N] ?

(Do you want to install Pi-specific nodes?)

As the installation progresses, the following will be displayed.

Running Node-RED update for user pi at /home/pi on raspbian


This can take 20-30 minutes on the slower Pi versions - please wait.

  Stop Node-RED                       ?
  Remove old version of Node-RED      ?
  Remove old version of Node.js       ?
  Install Node.js LTS                 ?   Node v12.16.0   Npm 6.13.7
  Clean npm cache                     ?
  Install Node-RED core               ?   1.0.3
  Move global nodes to local          -
  Install extra Pi nodes              -
  Npm rebuild existing nodes          -
  Add shortcut commands               ?
  Update systemd script               ?


Any errors will be logged to   /var/log/nodered-install.log
All done.
  You can now start Node-RED with the command  node-red-start
  or using the icon under   Menu / Programming / Node-RED
  Then point your browser to localhost:1880 or http://{your_pi_ip-address}:1880

Started Wednesday, February 19, 2020 15:45:42 JST  -Finished Wednesday, February 19, 2020 15:48:04 JST

7-2. Start Node-RED and make it a service

Start Node-RED with the following command.

Node-Launch RED


node-red

Use the following command to make Node-RED a service. By making it a service, Node-RED will be automatically executed when Raspberry Pi starts.

Node-RED service activation


sudo systemctl enable nodered.service

Service start(Automatic start only for the first time, after the next time)


sudo systemctl start nodered.service

7-3. Connect from a browser

With Node-RED running above, connect to the following URL from your browser.

http://<ip_address>:1880/

image.png

【Finally】

Although there are some issues, we were able to build the environment by following the procedure of the past. We have not checked the operation in detail yet, so please comment if you have any.

[Known bug]

I get an error when running ʻimport tensorflow` in python3

When I try to use TensorFlow and execute ʻimport tensorflow`, the following error is displayed.

2020-02-19 18:45:02.436414: E tensorflow/core/platform/hadoop/hadoop_file_system.cc:132] HadoopFileSystem load error: libhdfs.so: cannot open shared object file: No such file or directory

It has no effect so far, so I'm ignoring it at this time.

The version of TensorFlow that can be pip installed is old (as of 02/19/2020)

The latest version of TensorFlow that can be package-installed on Raspberry Pi is a little older, 1.14.0. For tf1 series, I would like to use the latest version 1.15.0. If you want to use the new version, you have to build it yourself. Official procedure of TensorFlow says that it can be built in about 30 minutes, but I tried several times, but it took more than 3 hours. And finally fail ...

As for the version of TensorFlow, @PINTO has published a pre-built package, which makes me very happy. At the moment, 1.15.0 is released for tf1 series, and 2.1.0 is released for tf2 series.

https://github.com/PINTO0309/Tensorflow-bin https://github.com/PINTO0309/Tensorflow-bin/#usage

In addition, if you use this package, the above error at the time of import will be solved.

Recommended Posts

Building a Raspberry Pi for studying Python and machine learning (RaspberryPi4 & Buster version (RaspberryPi3 is also possible))
Memo for building a machine learning environment using Python
How about Anaconda for building a machine learning environment in Python?
Building a Windows 7 environment for getting started with machine learning with Python
Why Python is chosen for machine learning
Building a Docker working environment for R and Python
Python learning memo for machine learning by Chainer Chapters 1 and 2
[Python] I made a classifier for irises [Machine learning]
Install pyenv on Raspberry Pi and version control Python
Raspberry Pi + python + IoT device, environment construction procedure to start image processing and machine learning
Installation procedure for Python and Ansible with a specific version
A beginner's summary of Python machine learning is super concise.
Building a Docker working environment for R and Python 2: Japanese support
<For beginners> python library <For machine learning>
[Definitive Edition] Building an environment for learning "machine learning" using Python on Mac
Quickly build a python environment for deep learning and data science (Windows)
Stream processing for Python and SensorTag, Kafka, Spark Streaming --Part 1: Raspberry Pi 3
The temperature is automatically measured using Raspberry Pi 3 and automatically uploaded to the server (Docker python3 + bottle + MySQL) for viewing!
[Raspberry Pi] Add a thermometer and a hygrometer
[Mac] Building a virtual environment for Python
[Shakyo] Encounter with Python for machine learning
[Python] Web application design for machine learning
Python and machine learning environment construction (macOS)
An introduction to Python for machine learning
Building a Python development environment for AI development
Creating a development environment for machine learning
I tried to explain what a Python generator is for as easily as possible.
Cross-compiling Raspberry Pi and building a remote debugging development environment with VS Code
Creating a temperature control system with Raspberry Pi and ESP32 (3) Recipient Python file
[For beginners] I made a motion sensor with Raspberry Pi and notified LINE!
Python learning notes for machine learning with Chainer Chapters 11 and 12 Introduction to Pandas Matplotlib