Display images taken with the Raspberry Pi camera module

I attached the camera module V2.1 to my Raspberry Pi 4, so I tried to create a simple program. I had a lot of trouble doing that, so I'll leave a note so I don't have to look it up again.

Programs using openCV

openCV has a function to capture the camera and also has a screen display function, which is very convenient.

test.py

import cv2
camera = cv2.VideoCapture(0)
camera.set(cv2.CAP_PROP_FRAME_WIDTH,640)
camera.set(cv2.CAP_PROP_FRAME_HEIGHT,480)
while True:
    _,image = camera.read()
#Line to add effect
    cv2.imshow('camera',image)
    key = cv2.waitKey(1)
    if key != -1:
        break
camera.release()
cv2.destroyAllWindows()

Pressing a key on the window will stop it.

Program description

The camera is initialized in the lower part. If you do not enter a specific value for the screen size, the image will be corrupted. Also, the argument of VideoCapture is the device ID, so it may change depending on the environment.

python

camera = cv2.VideoCapture(0)
camera.set(cv2.CAP_PROP_FRAME_WIDTH,640)
camera.set(cv2.CAP_PROP_FRAME_HEIGHT,480)

Taking images continuously is done in the lower part. The camera.read method puts new image data in frame every time the loop goes around.

python

while True:
    _,image = camera.read()

The window display is done in the lower part. It will not be displayed without the waitKey method.

python

    cv2.imshow('camera',image)
    key = cv2.waitKey(1)

The process of closing the opened window is done in the lower part.

python

camera.release()
cv2.destroyAllWindows()

Program using picamera and pygame

Only openCV is good, but considering the icon display and mouse operation on the same window, it seems more convenient to use pygame. (By the way, I tried to capture images in the picamera module.) There were many sites on the net that used the capture method, but this time I use the capture_continuous method.

test2.py

import picamera
from picamera.array import PiRGBArray
import pygame
import cv2                                                           #For effects
width = 480
height = 640
camera = picamera.PiCamera()
camera.resolution = (height,width)
pygame.init()
screen = pygame.display.set_mode((width,height))
rawCapture = PiRGBArray(camera,size=(height,width))
for frame in camera.capture_continuous(rawCapture,format='rgb',use_video_port=True):
    image = frame.array
#Line to add effect
    surf = pygame.surfarray.make_surface(image)
    screen.blit(surf,(0,0))
    pygame.display.flip()
    rawCapture.truncate(0)

To quit, press Ctrl + C to stop the program.

Program description

The camera is initialized in the lower part. The size of the image to be captured is determined by resolution, but for some reason the vertical and horizontal arrangements are reversed.

python

camera = picamera.PiCamera()
camera.resolution = (height,width)

The window display of pygame is done in the lower part.

python

pygame.init()
screen = pygame.display.set_mode((width,height))

Taking images continuously is done in the lower part. The capture_continuous method creates a generator and puts new image data in the frame every time the loop goes around. In addition, use_video_port = True aims at speeding up by extracting data directly from the video port. Also, new images cannot be captured without the final truncate method.

python

rawCapture = PiRGBArray(camera,size=(height,width))
for frame in camera.capture_continuous(rawCapture,format='rgb',use_video_port=True):
    ....
    ....
    rawCapture.truncate(0)

The description of pygame is done in the lower part. The image is made into an RGB 3D array (480,640,3) in the frame.array part. The pygame.surfarray.make_surface method converts it to the surface format that can be used for pygame, the blit method writes it to the screen, and the flip method updates the screen.

python

    image = frame.array
    surf = pygame.surfarray.make_surface(image)
    screen.blit(surf,(0,0))
    pygame.display.flip()

Addition of effects (common to openCV and picamera + pygame)

(Although the picamera module also has effects) Here, I will add the effects of openCV. You can change the program's #adding effect line to an edge detection image by changing it as shown below.

python

    image = cv2.Canny(image,100,200)

The image variable that receives the image data is in the RGB array format and can be calculated in the same way as the numpy array, so you can directly manipulate the RGB values ​​to edit the image.

at the end

I'm worried that the image loading is slow with either method. I would like to find out if there is a way to make it faster.