Even though I work in the non-IT department of the manufacturing industry, I have come to work with an awareness of AI and IoT. I work closer to production engineering jobs that are close to the manufacturing site, but one of the challenges in systematization is that it is not balanced with the profits to the factory. There is something I want to do, but if I estimate it, the introduction cost will be high (labor costs are often high ...) and I may give up. Therefore, the following content is what I am thinking that it is good to make it from the viewpoint of skill improvement.
We aim to process the recorded video used at the manufacturing site in real time (binarization, calculation / display of a certain distance, etc.) and display the result. Surprisingly, it can help on-site workers to easily manufacture even simple processed video images.
Well, it's been a long time, but the outline of this time is as follows.
The previous article is here.
Binarize the image. Furthermore, the shortest distance between two regions can be calculated (Ver1.1). https://qiita.com/Fumio-eisan/items/10c54af7a925b403f59f
First, perform the process of displaying the video. This time, we will process the video with the following capture for about 7 seconds.
video.ipynb
#Simply display
import cv2
import sys
file_path = 'sample_.mov'
delay = 1
window_name = 'frame'
cap = cv2.VideoCapture(file_path)
text = 'text.wmv'
if not cap.isOpened():
sys.exit()
while True:
ret, frame = cap.read()
# if not ret: #If you insert these two lines, the video will be played once.
# break
if ret:
frame = cv2.resize(frame, dsize=(600, 400))
cv2.imshow(window_name, frame)
if cv2.waitKey(delay) & 0xFF == ord('q'):
break
else:
cap.set(cv2.CAP_PROP_POS_FRAMES, 0)
cv2.destroyWindow(window_name)
It is OK if you describe the path of the video you want to display in file_path. In the case of this program, the video will continue to play in an infinite loop. You can stop the video playback by pressing the q key.
Regarding the display of the video, we are shooting according to the following procedure.
It has become. I will write a note later, but since it is processed every frame in 3., you need to be careful if you want to change the telop (value to be displayed, etc.) that you want to display every second or every few seconds.
video.ipynb
cv2.putText(frame, text,(100, 30), cv2.FONT_HERSHEY_SIMPLEX, 1.0, (0, 255, 0), thickness=2)
Subtitle output is possible with this display. The arguments are as follows. 1st argument: video frame, 2nd argument: text to be displayed, 3rd argument: x, y position, 4th argument: font, 5th argument: character size, 6th argument: BGR color information, 7th argument : Character thickness
Now, let's binarize it and display it in black and white. The method is OK if you define the upper and lower limits of the color to be extracted like the image and define it with the cv2.inRange () method.
video.ipynb
#Binarization process
import cv2
import sys
camera_id = 0
delay = 1
window_name = 'frame'
file_path = 'sample_.mov'
cap = cv2.VideoCapture(file_path)
import numpy as np
bgrLower = np.array([0, 100, 100]) #Lower limit of colors to extract(BGR)
bgrUpper = np.array([250,250, 250])
if not cap.isOpened():
sys.exit()
while True:
ret, frame = cap.read()
if not ret:
break
frame = cv2.resize(frame, dsize=(600, 400))
img_mask = cv2.inRange(frame, bgrLower, bgrUpper)
contours, hierarchy = cv2.findContours(img_mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours.sort(key=lambda x: cv2.contourArea(x), reverse=True)
#target_contour = max(contours, key=lambda x: cv2.contourArea(x))
#img_mask = cv2.line(img_mask, (250,300), (350,300), (120,120,120), 10) #The second argument is the start point, the third argument is the end point, the fourth argument is the color, and the fifth argument is the line thickness.
#img_mask=cv2.drawContours(img_mask,contours[0:2],-1,(120,120,120),5)
cv2.imshow(window_name, img_mask)
#cv2.imshow(window_name,img_mask, [contours[0],contours[1]])
if cv2.waitKey(delay) & 0xFF == ord('q'):
break
cv2.destroyWindow(window_name)
I was able to successfully binarize.
Well, this is the main subject. Define between the enclosed areas as shown below and find the distance between them. And I would like to output that value as appropriate.
video.ipynb
contours, hierarchy = cv2.findContours(img_mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)#Boundary drawing
contours.sort(key=lambda x: cv2.contourArea(x), reverse=True)#Sort boundaries
Enclose the white area with the cv2.findContours () method. The return value is stored in contours. Then sort the contours by area.
video.ipynb
x1=np.unravel_index(np.argmax(contours[0],axis=0), contours[0].shape)
x2=np.unravel_index(np.argmax(contours[1],axis=0), contours[0].shape)
img_mask = cv2.line(img_mask, tuple(x1[0][0]), tuple(x2[0][0]), (120,120,120), 3)
Gets the coordinates surrounding the area. Returns x1, x2 with argmax, which takes the maximum value of the coordinates in contours [0], [1](the value where either x or y is the maximum). In the case of argmax, it is flattened to one dimension (determined in one dimension regardless of x and y coordinates), so the unravel_index () method returns the index as coordinates.
Then, actually insert the coordinates with the cv2.line () method to connect the coordinates.
(Supplement) Understand the numerical values stored in contours
It's complicated like this.
Now, let's display this calculated value on the screen. Normally, it can be displayed by the cv2.putText () method. However, if it is left as it is, the value for each frame will be calculated and displayed. This will make the value flickering and hard to see. As a countermeasure, you can update the value every certain number of frames and display it. The following is done so that the value is updated in 30 frames (≈ about every second this time) using the if syntax.
video.ipynb
if idx % 30 == 0:
text =str(math.floor(np.linalg.norm(x1[0][0]-x2[0][0])))
cv2.putText(img_mask, text,(300, 100), cv2.FONT_HERSHEY_SIMPLEX, 1.0, (255, 255, 255), thickness=3)
Here is the result of processing and outputting in this way.
Originally, I wish I could give a value to white or black for each of the two areas of white and black, but I couldn't adjust it well. In addition, a line was drawn between the areas of the adapter, which is the white part (this time it is one thing, but it is divided into two areas), and it was also possible to output a numerical value indicating the distance.
Well, this time I made a program to play while processing the video. The point was that it was not possible to perform heavy processing because it was processed frame by frame. Originally, I wanted to find the shortest distance between regions and subtract that distance, but I couldn't implement it that much. Also, since there were many articles in which OpenCV itself was summarized in c ++, I had a hard time searching. ..
The program is stored below. https://github.com/Fumio-eisan/movie_20200406
Recommended Posts