Such a guy. A 400x400px image and a resized and rotated image of 200x200px are detected and matched with feature points.
This is the code that outputs the above image. only this.
import cv2
from IPython.display import Image
from IPython.display import display
#Image loading
img1 = cv2.imread('/path/to/dir/megane400x400.png')
img2 = cv2.imread('/path/to/dir/megane200x200_rotate.png')
#Feature point detection
akaze = cv2.AKAZE_create()
kp1, des1 = akaze.detectAndCompute(img1, None)
kp2, des2 = akaze.detectAndCompute(img2, None)
#matching
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True)
matches = bf.match(des1, des2)
#Sort by Hamming distance between feature points
matches = sorted(matches, key=lambda x: x.distance)
#Create matching result image between 2 images
img1_2 = cv2.drawMatches(img1, kp1, img2, kp2, matches[:10], None, flags=2)
decoded_bytes = cv2.imencode('.jpg', img1_2)[1].tobytes()
display(Image(data=decoded_bytes))
img1 = cv2.imread('/path/to/dir/megane400x400.png')
img2 = cv2.imread('/path/to/dir/megane200x200_rotate.png')
print(img1)
# [[[255 255 255]
# [255 255 255]
# [255 255 255]
# ...
# [255 255 255]
# [255 255 255]
# [255 255 255]]
#
# [[255 255 255]
# [255 255 255]
# [255 255 255]
# ...
# [255 255 255]
# [255 255 255]
# [255 255 255]]
#
# ...
#
# [[255 255 255]
# [255 255 255]
# [255 255 255]
# ...
# [255 255 255]
# [255 255 255]
# [255 255 255]]
#
# [[255 255 255]
# [255 255 255]
# [255 255 255]
# ...
# [255 255 255]
# [255 255 255]
# [255 255 255]]]
print(img1.shape)
# (400, 400, 3)
The BGR value per pixel is returned. In the cv2.imread () method, BGR (the order of the values is different) instead of RGB. Note that it needs to be converted to RGB when used with Pillow. See also: https://note.nkmk.me/python-opencv-bgr-rgb-cvtcolor/
This is a 400x400px image, so the shape is (400, 400, 3). [255 255 255] <-This is the BGR value per pixel. The white part of the image is where [255 255 255] are lined up.
akaze = cv2.AKAZE_create()
kp1, des1 = akaze.detectAndCompute(img1, None)
kp2, des2 = akaze.detectAndCompute(img2, None)
print('#####Feature point#####')
print(kp1)
# [<KeyPoint0x11af41db0>, <KeyPoint0x11af649c0>, <KeyPoint0x11af64ba0>,
# ...
# <KeyPoint 0x126265030>, <KeyPoint 0x126265120>, <KeyPoint 0x126265150>]
#The detected feature point is cv2.Returned as an array in the KeyPoint class.
print('#####Number of feature points#####')
print(len(kp1))
# 143
#Number of feature points Varies depending on the type and size of the image
#You can increase the number of feature points by enlarging the image, but if it exceeds a certain value, it will only increase the amount of calculation, so increase it while checking the output.
print('#####Feature descriptor#####')
print(des1)
# [[ 32 118 2 ... 253 255 0]
# [ 33 50 12 ... 253 255 48]
# [ 0 134 0 ... 253 255 32]
# ...
# [ 74 24 240 ... 128 239 31]
# [245 25 122 ... 255 239 31]
# [165 242 15 ... 127 238 55]]
#AKAZE returns as a 61-dimensional vector
print('#####Feature vector#####')
print(des1.shape)
# (143, 61) <- (58 is the number of feature points,Number of elements in feature descriptor)
AKAZE is one of the feature point detection algorithms and has the same standing position as ORB, SIFT, SURF, etc. It seems to have advantages such as fast calculation speed and easy to use because it is open source.
Also, according to this, AKAZE seems to have higher detection accuracy than ORB. https://docs.opencv.org/3.0-rc1/dc/d16/tutorial_akaze_tracking.html
Quote: http://labs.eecs.tottori-u.ac.jp/sd/Member/oyamada/OpenCV/html/py_tutorials/py_feature2d/py_features_meaning/py_features_meaning.html#features-meaning
A ~ F is a part cut out from this image. Do you know where the cropped image is? I think many people have this kind of impression.
A, B-> I understand that it is the sky and the wall, but it is difficult to specify the location. (flat) C, D-> You can see that it is somewhere in the upper part of the building. However, it is difficult to identify the exact location. (Edge) E, F-> You can easily see that it is the corner of the building. (corner)
From this, it seems that parts such as E and F are good features. In order to find corners such as E and F, algorithms such as AKAZE detect areas with large changes in brightness.
Reference: http://www.lab.kochi-tech.ac.jp/yoshilab/thesis/1150295.pdf
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True)
matches = bf.match(des1, des2)
matches = sorted(matches, key=lambda x: x.distance)
print(matches)
#Matched feature points are cv2.Returned as an array as a DMatch class
# [<DMatch 0x1260f5150>, <DMatch 0x1260f5490>, ... <DMatch 0x1260f65f0>, <DMatch 0x1260f69d0>]
print(len(matches))
# 58
print('###Distance between feature descriptors###')
for i in matches:
print(i.distance)
# 6.0
# 6.0
# .
# .
# .
# 142.0
# 150.0
BFMatcher brute force calculates the distance of feature descriptors (Hamming distance in this case) obtained from two images and matches the closest one. The first argument of BFMatcher (), cv2.NORM_HAMMING, specifies the Hamming distance to be calculated by distance. The default argument, crossCheck, is False, which creates an asymmetry in which one keypoint is closest, but the other keypoint is closer. True returns only the results that are the shortest on both sides.
Sort by return value by passing a function to the argument key of sorted ().
--matches-> List of DMatch type objects --DMatch.distance-> The lower the distance between feature descriptors, the higher the degree of matching. --DMatch.trainIdx-> Descriptor index in training descriptor (reference data) --DMatch.queryIdx-> Descriptor index in query descriptor (search data) --DMatch.imgIdx-> Index of training image
The number of different characters in the corresponding positions in two strings with the same number of characters.
The Hamming distance between 1011101 and 1001001 is 2. The Hamming distance between 2173896 and 2233796 is 3. -The Hamming distance between "toned" and "roses" is 3.
def1 and def2 passed to bf.match () are an array containing multiple 61-dimensional feature descriptors.
print(des1)
# [[ 32 118 2 ... 253 255 0] <-61 pieces
# [ 33 50 12 ... 253 255 48] <-61 pieces
# [ 0 134 0 ... 253 255 32]
# ...
# [ 74 24 240 ... 128 239 31]
# [245 25 122 ... 255 239 31]
# [165 242 15 ... 127 238 55]
It seems that the following processing is done in bf.match ().
You can see that it has been converted to decimal-> binary.
#0 is a binary number 00000000
des1 = np.array([0]).astype('uint8')
#255 is a binary number 11111111
des2 = np.array([255]).astype('uint8')
# ※ astype('uint8')If you don't, bf.match()I can't give it to you.
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True)
matches = bf.match(des1, des2)
matches = sorted(matches, key=lambda x: x.distance)
for i in matches:
print(i.distance)
# 8.0 <-Hamming distance
#244 is a binary number 11111110
des1 = np.array([244]).astype('uint8')
#255 is a binary number 11111111
des2 = np.array([255]).astype('uint8')
# ※ astype('uint8')If you don't, bf.match()I can't give it to you.
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True)
matches = bf.match(des1, des2)
for i in matches:
print(i.distance)
# 1.0 <-Hamming distance
You can see that we are summing the distances between the elements in the feature descriptor.
#Binary representation is possible by adding a 0b prefix.
des1 = np.array([[0b0001, 0b0001, 0b0001], [0b0011, 0b0011, 0b0011], [0b0111, 0b0111, 0b0111]]).astype('uint8')
des2 = np.array([[0b0000, 0b0000, 0b0000]]).astype('uint8')
# ※ astype('uint8')If you don't, bf.match()I can't give it to you.
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True)
matches = bf.match(des1, des2)
# [0b0001, 0b0001, 0b0001]When[0b0000, 0b0000, 0b0000]Only the result of
for i in matches:
print(i.distance)
# 3.0 <-Hamming distance
img1_2 = cv2.drawMatches(img1, kp1, img2, kp2, matches[:10], None, flags=2)
decoded_bytes = cv2.imencode('.jpg', img1_2)[1].tobytes()
display(Image(data=decoded_bytes)
By setting matches [:10] in the argument of drawMatches (), 10 feature points with close distances are drawn from the top.
Recommended Posts