Recognize the contour and direction of a shaped object with OpenCV3 and Python3 (Principal component analysis: PCA, eigenvectors)
What is Principal Component Analysis (PCA)?
There are various features of an object, and it may be possible to identify the type of object by the features. For example, coffee has features such as sourness, bitterness, roasting, richness, color, and aroma, and the combination can identify the type of coffee (mocha, kirimanjaro, mandelin, etc.). .. However, there are various features, and when identifying the type, some features are effective and some are not very effective. For example, weight and hardness can be considered as features of coffee, but this is unlikely to be very useful for identifying the type. The price may be a little useful as a feature.
Principal component analysis (PCA) is an analysis method that uses standard deviation, covariance matrix, eigenvectors, etc. to identify strongly characteristic components when identifying an object. In addition, by paying attention to the features that are closely related, it is possible to perform dimensional compression of the data, which is effective in reducing the amount of calculation and the amount of data / memory.
OpenCV also supports Principal Component Analysis (PCA) in the core module. This time, I will create a program that calculates the direction of a shaped object using a method that handles Principal Component Analysis (PCA) of OpenCV and displays the outline and the direction of the principal component with arrows. The final result should look like the figure below.
** Display the outline (red) and principal component vector (light blue) of the object **
OpenCV OpenCV (Open Source Computer Vision Library) is a collection of BSD-licensed video / image processing libraries. There are many algorithms for image filtering, template matching, object recognition, video analysis, machine learning, and more.
■ Example of motion tracking using OpenCV (OpenCV Google Summer of Code 2015) https://www.youtube.com/watch?v=OUbUFn71S4s
■ Click here for installation and easy usage Install OpenCV 3 (core + contrib) in Python 3 environment & Difference between OpenCV 2 and OpenCV 3 & simple operation check
■ Click here for still image processing Try edge detection with OpenCV Perform various filters with OpenCV (gradient, high-pass, Laplacian, Gaussian) Extract feature points with OpenCV (AgastFeature, FAST, GFTT, MSER, AKAZE, BRISK, KAZE, ORB, SimpleBlob) Face recognition using OpenCV (Haar-like feature classifier) Estimate who's face using OpenCV (Eigenface, Fisherface, LBPH)
■ Click here for video processing Try converting videos in real time with OpenCV Try converting webcam / camcorder video in real time with OpenCV Draw optical flow in real time with OpenCV (Shi-Tomasi method, Lucas-Kanade method) Object tracking using OpenCV (tracking feature points specified by mouse by Lucas-Kanade method Motion template analysis using OpenCV (recognizes objects and their moving directions in real time)
algorithm
Usually, when it comes to principal component analysis, you often see the following points in the scatter plot. The plot is a two-dimensional compression of features that were originally multidimensional. In addition, the arrows represent the vectors of the first and second principal components for the plot.
Source: Wikipedia-Principal component analysis
Principal component analysis of figures can also be performed with this application. The procedure is as follows.
- Converts color information that seems unnecessary to recognize the direction of an object into black and white information.
- Recognize the outline of the object as a two-dimensional (x-coordinate, y-coordinate) point.
- Principal component analysis (PCA) is performed on a collection of contour points (x, y).
- Find the vector (eigenvector) in the direction of the principal component and draw it on the image.
- Repeat steps 1 to 4 for each object.
- In the coffee example, richness and bitterness were the features, but in the image, the x and y coordinates are the features.
- In step 1, RGB color information is converted to black and white information. It's not a PCA analysis, but it's also a type of dimensional compression.
- Step 2 seems to be difficult as it is, but for this, OpenCV's cv2.findContours () method calculates the coordinates of the contour for each object.
- For step 3, you can use OpenCV's cv2.PCACompute () method to find the mean and eigenvectors (vectors in the principal component direction).
- For step 4, draw a line in the eigenvector direction with OpenCV's cv2.line () method.
So, if you use OpenCV, you can recognize the outline and direction of an object just by calling a few methods.
program
-
Execution environment
- Python: ver.3.5.2
- OpenCV: ver.3.1.0
-
Sample image --Use the image (pca_test1.jpg) that comes with the OpenCV package. [OpenCV Install Dir]/samples/data/pca_test1.jpg
-
Program notes --Drawing system is processed by int --Analytical system is processed by float If you execute the PCACompute () method with the contents of the array int, the following error will occur.
``` error: (-210) Unsupported combination of input and output array formats in function cv::reduce ```
--Instructed to acquire only one dimension of the main component (maxComponents = 1) --Instructed to acquire the data of the midpoint because the coordinates of the contour are used for the principal component analysis (cv2.CHAIN_APPROX_NONE). --Usually, when performing principal component analysis, data is standardized as preprocessing. In this image processing, there is no difference in the weighting of the two-dimensional x-coordinate and y-coordinate, so no pre-processing for data standardization is performed.
pca.py
# -*- coding: utf-8 -*-
import cv2
import math
import numpy as np
#Draw vector
def drawAxis(img, start_pt, vec, colour, length):
#Antialiasing
CV_AA = 16
#End point
end_pt = (int(start_pt[0] + length * vec[0]), int(start_pt[1] + length * vec[1]))
#Draw center
cv2.circle(img, (int(start_pt[0]), int(start_pt[1])), 5, colour, 1)
#Draw axis
cv2.line(img, (int(start_pt[0]), int(start_pt[1])), end_pt, colour, 1, CV_AA);
#Draw the arrow at the tip
angle = math.atan2(vec[1], vec[0])
print(angle)
qx0 = int(end_pt[0] - 9 * math.cos(angle + math.pi / 4));
qy0 = int(end_pt[1] - 9 * math.sin(angle + math.pi / 4));
cv2.line(img, end_pt, (qx0, qy0), colour, 1, CV_AA);
qx1 = int(end_pt[0] - 9 * math.cos(angle - math.pi / 4));
qy1 = int(end_pt[1] - 9 * math.sin(angle - math.pi / 4));
cv2.line(img, end_pt, (qx1, qy1), colour, 1, CV_AA);
if __name__ == '__main__':
#Load image
src = cv2.imread("pca_test1.jpg ")
#Convert to grayscale
gray = cv2.cvtColor(src, cv2.COLOR_BGR2GRAY)
#Binarization
retval, bw = cv2.threshold(gray, 50, 255, cv2.THRESH_BINARY | cv2.THRESH_OTSU)
#Extract contour
# contours : [region][Point No][0][x=0, y=1]
# cv2.CHAIN_APPROX_NONE:Hold the midpoint
# cv2.CHAIN_APPROX_SIMPLE:Do not hold midpoint
img, contours, hierarchy = cv2.findContours(bw, cv2.RETR_LIST, cv2.CHAIN_APPROX_NONE)
#Processing for each contour
for i in range(0, len(contours)):
#Calculate contour area
area = cv2.contourArea(contours[i])
#Exclude noise (too small area) and overall contour (too large area)
if area < 1e2 or 1e5 < area:
continue
#Draw contour
cv2.drawContours(src, contours, i, (0, 0, 255), 2, 8, hierarchy, 0)
#Store contour data in a floating point array
X = np.array(contours[i], dtype=np.float).reshape((contours[i].shape[0], contours[i].shape[2]))
#PCA (1D)
mean, eigenvectors = cv2.PCACompute(X, mean=np.array([], dtype=np.float), maxComponents=1)
#Draw vector in direction of principal component
pt = (mean[0][0], mean[0][1])
vec = (eigenvectors[0][0], eigenvectors[0][1])
drawAxis(src, pt, vec, (255, 255, 0), 150)
#display
cv2.imshow('output', src)
cv2.waitKey(0)
#End processing
cv2.destroyAllWindows()
The execution result will look like the image shown at the beginning.
bonus
Let's recognize the contour and direction not only for the sample image but also for other images.
For example, let's analyze by placing a hanger with a characteristic hole on a pattern like a tatami mat.
As a result, we were able to display the vector in the direction of the principal component for the three contours surrounding the small hole, the outer contour and the inner contour.
Recommended Posts