Control the Matrix LED panel from ROS
Introduction
I bought a Matrix LED (64x64) from Shigezone in Akihabara, but I decided to run it on ROS. In this article, we will create a "ROS node that subscribes to the Image topic and displays the image on the Matrix LED".
Environmental preparation
Preparing the Raspberry Pi
This time, we will use Raspberry Pi 3 with Ubuntu installed to handle ROS. Flash the image of here to the SD card and install ROS. I have installed Ubuntu 18.04.3 on arm64. ROS installed ros-melodic-desktop by referring to this site.
Preparation of Matrix LED panel connection board
In order to use this software as a driver for MatrixLED, we have prepared a corresponding driver board. The board was [obtained] from electrodragon (https://www.electrodragon.com/product/rgb-matrix-panel-drive-board-raspberry-pi/), but you may prepare it yourself. The board you purchased has a built-in 3.3V to 5V level converter, so you can expect stable operation. Since the Matrix LED used is 64x64 in size and has address lines A to E, it is necessary to solder the driver board side in one place to connect "E". For the Matrix LED panel I purchased this time, I set a jumper to connect Pin8 to E. It is necessary to supply 5V (I think it is at least 3A) separately to the LED panel body.
LED panel driver preparation
To see if the hardware works properly, clone from the repository and then compile the sample program.
rpi-rgb-led-matrix$ make -C examples-api-use
rpi-rgb-led-matrix$ sudo examples-api-use/demo -D0 --led-rows=64 --led-cols=64 --led-panel-type=FM6126A
The colorful cubes should spin around. Depending on the type of Matrix LED panel, FM6126A may be mounted as a chip. In that case, the signal required for initialization is different, so it is necessary to specify the option.
If it works, install the library for Python according to Procedure of the repository. I will.
$ sudo apt-get update && sudo apt-get install python2.7-dev python-pillow -y
rpi-rgb-led-matrix/bindings/python$ make build-python
rpi-rgb-led-matrix/bindings/python$ sudo make install-python
At this point, you will be able to do "from rgbmatrix import RGBMatrix, RGBMatrixOptions".
To use ROS with sudo
The Matrix LED driver software used this time must be run with sudo. With the combination of Ubuntu18.04 + Raspberry pi3, I could not access the hardware resources (GPIO etc.) required by general users, so I decided to run the ROS side with sudo. (I think it should be set so that general users can access the necessary hardware resources.)
In general, ROS doesn't work because environment variables are cleared when sudo is run. So I decided to add the settings to the sudoers file.
Defaults env_keep += "PATH PKG_CONFIG_PATH PYTHONPATH ROS_ROOT ROS_ETC_DIR ROS_MASTER_URI ROS_VERSION ROS_PYTHON_VERSION ROS_PACKAGE_PATH ROS_DISTRO CMAKE_PREFIX_PATH LD_LIBRARY_PATH"
By adding this one line with the "sudo visudo" command, the environment variables required for ROS will be inherited when sudo is executed. However, LD_LIBRARY_PATH is not inherited by the specifications.
Therefore, I decided to link the required path in "/etc/ld.so.conf.d/ros.conf". In the case of ROS merodic, it looks like this.
/home/user/catkin_ws/devel/lib
/opt/ros/melodic/lib
After completing the settings, execute "sudo ld config".
In addition, set permissions so that daemon users can access /home/user/.ros/log. Without this, it will stop with a Permission error when executing a ROS node. (When running a ROS node with sudo, the Log file seems to be generated as the daemon user.) At this point, I think you can access ROS resources even with sudo.
Implementation of ROS node
First, make a ROS package.
~/catkin_ws/src$ catkin_create_pkg matrix_led_ros roscpp rospy std_msgs
~/catkin_ws$ catkin_make
From here, we will implement the script under the created matrix_led_ros package.
Create a node to publish an Image topic from a GIF
In order to test the program to be created next, I implemented a node that reads a GIF and publishes it as an Image topic.
#!/usr/bin/env python
from __future__ import print_function
import roslib
import sys
import rospy
import cv2
from std_msgs.msg import String
from sensor_msgs.msg import Image
from cv_bridge import CvBridge, CvBridgeError
import argparse
FILE_NAME = "test.gif"
REFRESH_RATE = 1
parser = argparse.ArgumentParser(
prog='gif-publisher.py',
usage='python gif-publisher --filename inputfilename',
description='publish Image message from gif file',
epilog='end',
add_help=True,
)
parser.add_argument('-f', '--filename', help='input file name',
required=True)
parser.add_argument('-r', '--rate', help='refresh rate',type=int)
args = parser.parse_args()
if args.filename:
FILE_NAME = args.filename
if args.rate:
REFRESH_RATE = args.rate
class image_publisher:
def __init__(self):
self.image_pub = rospy.Publisher("/imagetopic",Image)
self.bridge = CvBridge()
def readGif(self, filename, hz=1):
gif = cv2.VideoCapture(filename)
r = rospy.Rate(hz)
while not rospy.is_shutdown():
try:
stat, frame = gif.read()
if not stat:
gif = cv2.VideoCapture(filename)
else:
try:
self.image_pub.publish(self.bridge.cv2_to_imgmsg(frame, "bgr8"))
r.sleep()
except CvBridgeError as e:
print(e)
except KeyboardInterrupt:
break
def main(args):
ip = image_publisher()
rospy.init_node('gif_image_publisher', anonymous=True)
ip.readGif(FILE_NAME, REFRESH_RATE)
try:
rospy.spin()
except KeyboardInterrupt:
print("Shutting down")
if __name__ == '__main__':
main(sys.argv)
Use it like "python gif-publisher.py -f test.gif -r 10". This example publishes test.gif at 10fps with the name / imagetopic. If nothing is specified, the frame will be published at explosive speed, so I set the frame rate with "r = rospy.Rate (hz)" and "r.sleep ()". In the case of Raspberry pi, it worked up to less than 30fps.
Create a ROS node that receives the Image topic
It's finally the main subject. A script that subscribes to / imagetopic and displays it on the Matrix LED panel.
#!/usr/bin/env python
from __future__ import print_function
import roslib
import sys, time
import rospy
import cv2
from std_msgs.msg import String
from sensor_msgs.msg import Image
from cv_bridge import CvBridge, CvBridgeError
from rgbmatrix import RGBMatrix, RGBMatrixOptions
from PIL import Image as pilI
BRIGHTNESS = 0.5
class image_viewer:
def __init__(self):
self.bridge = CvBridge()
self.image_sub = rospy.Subscriber("imagetopic",Image,self.callback)
# Configuration for the matrix
self.options = RGBMatrixOptions()
self.options.rows = 64
self.options.cols = 64
self.options.chain_length = 1
self.options.parallel = 1
self.options.hardware_mapping = 'regular'
self.matrix = RGBMatrix(options = self.options)
self.max_brightness = self.matrix.brightness
self.matrix.brightness = self.max_brightness * BRIGHTNESS
self.double_buffer = self.matrix.CreateFrameCanvas()
def toSquare(self, img):
w, h =img.size
if w == h:
return img
elif w > h:
result = pilI.new(img.mode, (w, w), (0,0,0))
result.paste(img, (0, (w - h) // 2))
return result
else:
result = pilI.new(img.mode, (h, h), (0,0,0))
result.paste(img, ((h - w) // 2, 0))
return result
def callback(self,data):
try:
cv_image = self.bridge.imgmsg_to_cv2(data, "bgr8")
except CvBridgeError as e:
print(e)
pilImage = cv2.cvtColor(cv_image, cv2.COLOR_BGR2RGB)
pilImage = pilI.fromarray(pilImage)
pilImage.thumbnail((self.matrix.width, self.matrix.height), pilI.ANTIALIAS)
offset = (int)(self.matrix.height - pilImage.height)/2
self.double_buffer.SetImage(self.toSquare(pilImage), 0)
self.double_buffer = self.matrix.SwapOnVSync(self.double_buffer)
def main(args):
iv = image_viewer()
rospy.init_node('image_viewer')
try:
rospy.spin()
except KeyboardInterrupt:
print("Shutting down")
if __name__ == '__main__':
main(sys.argv)
Start it like "sudo python image-viewer-ros.py". This script will not work well unless you have root privileges, but you can make it work with ROS in sudo by the contents set in "To use ROS with sudo" earlier.
It looks like this when it moves. I borrowed this GIF animation. 
In the "self.matrix = RGBMatrix (options = self.options)" section, you can specify the options to pass to the matrix LED. For images larger than 64x64, the long sides are reduced to 64 pixels and the margins are filled with black.
At the end
What did you think. In this implementation, there is a part where flicker is a little worrisome, so there seems to be room for improvement. If I have a chance, I will also list the one implemented in C ++. In addition, the driver board used this time can use 3 channels at the same time, and as a Matrix LED, it can be expanded by connecting beads, so if you can get multiple LED panels, I would like to try higher resolution.
Recommended Posts