Easy TopView with OpenCV
Suddenly I got a Raspberry Pi Zero W.
I was lucky enough to order from Switch Science on the release date and it was available immediately! I immediately installed this on Plarail and succeeded in operating it well. We have published the code for mounting on Plarail and remote control with Wifi on the following Git, so please have a look if you are interested. https://github.com/Kumapapa2012/Raspberry-Pi-Zero-on-PLA-RAIL-3rd
About Top View
About Top View of the main subject. Top View is a technology that makes the image in the field of view of the camera look down as if it were looking down from directly above. It is also installed in a certain company's car.
When thinking about automatic driving by attaching a camera to something that moves around, not limited to Plarail, I think that I am quite worried about how to do it with a monocular camera. For example, how should you determine your own direction of travel and speed? Should I find a feature point between a frame at a certain point in time and a frame after a certain period of time, and estimate my own movement from the movement position, scaling ratio, etc.? etc. .. .. Meanwhile, I found an article that Top View could easily solve the problem. [^ 1]
The procedure is simple.
-
Take a picture of a suitable square or rectangle aligned with the bottom of the screen and the center of the left and right, and get the coordinates of the vertices on the screen. The image is as follows.
This image is taken with the letter "A" in a 4 cm square.
In this case, the coordinates are as follows in the order of upper left, upper right, lower right, lower left.
T_L=(168,260)
T_R=(344,260)
B_R=(408,360)
B_L=(72,360) -
Determine the image size when converted to TopView, how a square or rectangle should be displayed in the image, and determine the vertex coordinates. It uses the longest or base of the original "A" image and calculates the coordinates so that they are all squares of the same length. See my Notebook [^ 2] for more details.
-
Use cv2.getPerspectiveTransform () to get the transformation matrix using each vertex coordinate. This transformation matrix transforms the trapezoidal "A" on the camera into the square "A" seen from above. Perhaps this transformation matrix should be calculated once as a constant term.
4.cv2.warpPerspective () Apply the transformation matrix obtained in 3. to the camera image.
Now you have a Top View.
I uploaded a video of the actual operation on Youtube. In this Video, the Top View effect is applied to the camera image edge-extracted by the "Canny" algorithm. Although it is a little distorted, "A" appears in a square shape.
- Click to play
There is a delay with the video in Notebook, probably because you forced Notebook to display the video.
Although it still needs to be adjusted, Top View will make the environmental awareness problem a 3D to 2D problem, and the classifier requirements will likely be much simpler. If the rails can be identified from the Top View and can be tracked, it seems easy to detect obstacles and stop, for example.
References
Line detection autopilot using Python + OpenCV. Canny Edge Detection Display OpenCV images inline with Jupyter
Recommended Posts