Feature matching with OpenCV 3 and Python 3 (A-KAZE, KNN)

Feature matching with OpenCV

Feature matching is the matching of features extracted from different images.

It is a technology that appears in.

The following libraries are provided in OpenCV.

This time, using OpenCV 3 + Python 3, we will try to match the features of the rotated and zoomed image as shown below.

match_thumbnail.jpg ** Draw matching result **

What is OpenCV

OpenCV (Open Source Computer Vision Library) is a collection of BSD-licensed video / image processing libraries. There are many algorithms for image filtering, template matching, object recognition, video analysis, machine learning, and more.

■ Example of motion tracking using OpenCV (OpenCV Google Summer of Code 2015) https://www.youtube.com/watch?v=OUbUFn71S4s

■ Click here for installation and easy usage Install OpenCV 3 (core + contrib) in Python 3 environment & Difference between OpenCV 2 and OpenCV 3 & simple operation check

■ Click here for still image processing Try edge detection with OpenCV Perform various filters with OpenCV (gradient, high-pass, Laplacian, Gaussian) Extract feature points with OpenCV (AgastFeature, FAST, GFTT, MSER, AKAZE, BRISK, KAZE, ORB, SimpleBlob) Face recognition using OpenCV (Haar-like feature classifier) Estimate who's face using OpenCV (Eigenface, Fisherface, LBPH) Recognize the contour and direction of a shaped object with OpenCV (principal component analysis: PCA, eigenvectors)

■ Click here for video processing Try converting videos in real time with OpenCV Try converting webcam / camcorder video in real time with OpenCV Draw optical flow in real time with OpenCV (Shi-Tomasi method, Lucas-Kanade method) Object tracking using OpenCV (tracking feature points specified by mouse by Lucas-Kanade method Motion template analysis using OpenCV (recognizes objects and their moving directions in real time)

A-KAZE

KNN、Brute-Force、FLANN KNN (K-Nearest Neighbor algorithm) is an algorithm that selects K nearest neighbor labels from the search space and assigns class labels by majority vote. Learning simply stores the training data as it is. It operates at high speed because the learning cost is zero. A representative player of the laziness learning algorithm. OpenCV supports the brute-force method (Brute-Force) and the fast approximate neighbor search method (FLANN) as a method for searching K nearest neighbor labels from the search space.

● Remarks (Python3 and FLANN)

Earlier I wrote that there is a tutorial that doesn't work on OpenCV 3 (link), but FLANN feature point matching is my environment (OpenCV) It didn't work on 3.1.0 + Python 3.5.2 + Windows 10).

    > matches = flann.knnMatch(des1,des2,k=2)
    > 

\ # The following error occurs # error: C:\dev\opencv-3.1.0\modules\python\src2\cv2.cpp:163: error: (-215) The data should normally be NULL! in function NumpyAllocator::allocate

FLANN works fine in C ++ and OpenCV2, so if you want to use FLANN, run it in C ++ or OpenCV2 environment. When working with a combination of OpenCV 3 and Python 3, currently the "brute-force method" is used.

program

knn.py


# -*- coding: utf-8 -*-
import cv2

#Image 1
img1 = cv2.imread("img1.jpg ")
#Image 2
img2 = cv2.imread("img2.jpg ")

# A-Generation of KAZE detector
akaze = cv2.AKAZE_create()                                

#Feature detection and feature vector calculation
kp1, des1 = akaze.detectAndCompute(img1, None)
kp2, des2 = akaze.detectAndCompute(img2, None)

# Brute-Force Matcher generation
bf = cv2.BFMatcher()

#Brute feature vectors-Matching with Force & KNN
matches = bf.knnMatch(des1, des2, k=2)

#Thin out data
ratio = 0.5
good = []
for m, n in matches:
    if m.distance < ratio * n.distance:
        good.append([m])

#Draw corresponding feature points
img3 = cv2.drawMatchesKnn(img1, kp1, img2, kp2, good[], None, flags=2)

#Image display
cv2.imshow('img', img3)

#Press the key to finish
cv2.waitKey(0)
cv2.destroyAllWindows()

Matching result

The Ratio Test proposed by D. Lowe thins out the matching results and displays them.

lowe.py


#Thin out data
ratio = 0.5
lowe = []
for m, n in matches:
    if m.distance < ratio * n.distance:
        lowe.append([m])

#Draw corresponding feature points
img3 = cv2.drawMatchesKnn(img1, kp1, img2, kp2, lowe[], None, flags=2)

This method gives you a visual idea of the matching status of the entire image. There seems to be no false positives in this test.

match_result.png

Let's change a part of the code so that only those with good matching status between features are displayed.

knn_good.py


#Sort features according to matching status
good = sorted(matches, key = lambda x : x[1].distance)

#Draw corresponding feature points
img3 = cv2.drawMatchesKnn(img1, kp1, img2, kp2, good[:30], None, flags=2)

match_good.png

A different feature than before was drawn. This time, since the features are dispersed, they match as a whole, but in the case of an image in which the feature points are partially biased, those with good matching conditions will be concentrated there. ..

KeyPoint (feature point object)

attribute Contents
pt Point (x, y)
size Feature point diameter
angle [0, 360)The angle of the range. The y-axis is downward and clockwise. If it cannot be calculated-1。
response Strength of feature points
octave Pyramid layer that detected feature points
class_id ID of the class to which the feature point belongs

DMatch (matching result object)

The following items are stored in DMatch as a result of matching between features.

attribute Contents
distance Distance between features. The closer the distance, the better the match.
trainIdx Training feature index
queryIdx Query feature index
imgIdx Training image index

Recommended Posts

Feature matching with OpenCV 3 and Python 3 (A-KAZE, KNN)
Shining life with Python and OpenCV
Neural network with OpenCV 3 and Python 3
Binarization with OpenCV / Python
Install OpenCV 4.0 and Python 3.7 on Windows 10 with Anaconda
Extract feature points with OpenCV3 and Python3 (AgastFeature, FAST, GFTT, MSER, AKAZE, BRISK, KAZE, ORB, SimpleBlob, SIFT)
[Python] Webcam frame size and FPS settings with OpenCV
Reading, displaying and speeding up gifs with python [OpenCV]
Programming with Python and Tkinter
Encryption and decryption with Python
"Apple processing" with OpenCV3 + Python3
Python and hardware-Using RS232C with Python-
Image editing with python OpenCV
Camera capture with Python + OpenCV
[Python] Using OpenCV with Python (Basic)
Face detection with Python + OpenCV
python with pyenv and venv
Using OpenCV with Python @Mac
Works with Python and R
Convert video to black and white with ffmpeg + python + opencv
Communicate with FX-5204PS with Python and PyUSB
Environment construction of python and opencv
Robot running with Arduino and python
Install Python 2.7.9 and Python 3.4.x with pip.
AM modulation and demodulation with python
[Python] font family and font with matplotlib
Scraping with Node, Ruby and Python
[Python] Using OpenCV with Python (Image transformation)
Scraping with Python, Selenium and Chromedriver
[Python] Using OpenCV with Python (Edge Detection)
Scraping with Python and Beautiful Soup
JSON encoding and decoding with python
Hadoop introduction and MapReduce with Python
Reading and writing NetCDF with Python
Try face recognition with python + OpenCV
Cut out face with Python + OpenCV
Face recognition with camera with opencv3 + python2.7
OpenCV feature detection with Google Colaboratory
Load gif images with Python + OpenCV
Reading and writing CSV with Python
Find image similarity with Python + OpenCV
Use OpenCV with Python 3 in Window
Multiple integrals with Python and Sympy
Draw an illustration with Python + OpenCV
Coexistence of Python2 and 3 with CircleCI (1.0)
Track baseball balls with Python + OpenCV
Easy modeling with Blender and Python
Graph Based Segmentation with Python + OpenCV
Draw shapes with OpenCV and PIL
Sugoroku game and addition game with python
FM modulation and demodulation with Python
Draw arrows (vectors) with opencv / python
Basic study of OpenCV with Python
Automatic image interpolation with OpenCV and Python (Fast Marching Method, Navier-Stokes)
Create a striped illusion with gamma correction for Python3 and openCV3
How to make a surveillance camera (Security Camera) with Opencv and Python
Make a simple OMR (mark sheet reader) with Python and OpenCV
Draw a watercolor illusion with edge detection in Python3 and openCV3
Object extraction in images by pattern matching using OpenCV with Python
Make blur videos look like fixed-point cameras with Python and OpenCV
[Python] Try to recognize characters from images with OpenCV and pyocr