How to save the feature point information of an image in a file and use it for matching

Introduction

There are many opportunities to use OpenCV feature point extraction and matching. When using multiple images, it often took time to extract feature points. Therefore, this time, I made a program that outputs the feature point information to a file and uses it for matching.

This time, as an example, let's find a girl with red glasses (target image) from the girls (12 source images)!

Source image ⬇︎ youngwoman_37.jpgyoungwoman_38.jpgyoungwoman_39.jpgyoungwoman_40.jpg

youngwoman_41.jpgyoungwoman_42.jpgyoungwoman_43.jpgyoungwoman_44.jpg

youngwoman_45.jpgyoungwoman_46.jpgyoungwoman_47.jpgyoungwoman_48.jpg

Target image ⬇︎ target_girl.jpg

Development environment

This time, we developed in the following environment. OpenCV 4.1.2 Python 3.8.1

Constitution

Below is the flow of the program. I will explain step by step from the next section.

  1. Get the feature point information of the source image and save it in a file
  2. Get feature point information of target image
  3. Get the feature point information of the source image from the file
  4. Matching

1. Get the feature point information of the source image and save it in a file

First, get the feature point information of the source image (girls) in save_features.py and save it in a file. This time, I used AKAZE, a feature point descriptor implemented in OpenCV. To save the keypoint as a file, you need to access and list the cv :: KeyPoint.

save_features.py


#List keypoint
keypoint = []
for p in features[0]:
    temp = (p.pt, p.size, p.angle, p.response, p.octave, p.class_id)
    keypoint.append(temp)

In addition, this time, the feature point information was converted to byte type in order to reduce memory consumption.

save_features.py


#Convert keypoint to bytes
map(bytes, keypoints)

Below is the entire source code.

save_features.py


import cv2 as cv
import pickle

SOURCE_FILES = [
    "youngwoman_37.jpg ",
    "youngwoman_38.jpg ",
    "youngwoman_39.jpg ",
    "youngwoman_40.jpg ",
    "youngwoman_41.jpg ",
    "youngwoman_42.jpg ",
    "youngwoman_43.jpg ",
    "youngwoman_44.jpg ",
    "youngwoman_45.jpg ",
    "youngwoman_46.jpg ",
    "youngwoman_47.jpg ",
    "youngwoman_48.jpg ",
]


def get_features(img_file_name):
    """Get features of master images

        Args:
            img_file_name(list): Master image

        Returns:
            keypoints, descriptors, img

    """
    img = cv.imread("images/" + img_file_name)

    #Feature point information extraction
    akaze = cv.AKAZE_create()
    kp, des = akaze.detectAndCompute(img, None)

    features = [kp, des]

    return features


sources = {}
for item in SOURCE_FILES:
    features = get_features(item)
    #List keypoint
    keypoints = []
    for p in features[0]:
        temp = (p.pt, p.size, p.angle, p.response, p.octave, p.class_id)
        keypoints.append(temp)

    #Convert keypoints to bytes
    map(bytes, keypoints)
    #Dictionary of feature point information
    sources[item] = {
        "src": item,
        "keypoint": keypoints,
        "descriptor": features[1],
    }

#Write feature point information to a file
with open("sources_data.pickle", mode="wb") as f:
    pickle.dump(sources, f)

2. Get feature point information of target image

From step 2, get_features_from_file.py is used for processing. As with the source image, feature point information is acquired using AKAZE, which is a feature point descriptor.

get_features_from_file.py


#Load target image
target_img = cv.imread("images/target_girl.jpg ")
#Feature acquisition
akaze = cv.AKAZE_create()
target_keypoint, target_descriptor = akaze.detectAndCompute(target_img, None)

3. Get the feature point information of the source image from the file

Get feature point information from the file with get_sources (). Since keypoints was converted to bytes in order to make it pickle, it is converted to list and returned to the original structure.

get_features_from_file.py


def get_sources():
    """Get source's features from file

        Returns:
            sources(list): source's keypoints, descriptors,and img
        """
    #Get feature point information from a file
    with open("sources_data.pickle", mode="rb") as f:
        sources = pickle.load(f)

    for n in sources:
        items = sources[n]
        #Change keypoints from bytes to list
        list(map(list, items["keypoint"]))
        #Restore keypoints to original structure
        keypoints = []
        for p in items["keypoint"]:
            temp = cv.KeyPoint(
                x=p[0][0],
                y=p[0][1],
                _size=p[1],
                _angle=p[2],
                _response=p[3],
                _octave=p[4],
                _class_id=p[5],
            )
            keypoints.append(temp)
        items["keypoint"] = keypoints

    return sources

4. Matching

Matches with the feature point information of the target image for each source image. Data is thinned out, and if the number of matched feature points is equal to or greater than the set threshold (set to 20 this time), matching is successful.

get_features_from_file.py


for n in sources:
    source = sources[n]
    source_img = cv.imread("images/" + source["src"])
    matches = matcher.knnMatch(source["descriptor"], target_des, k=2)
    #Thin out data
    ratio = 0.5
    matched_keypoints = []
    for m, n in matches:
        if m.distance < ratio * n.distance:
            matched_keypoints.append([m])

    #Output the result image when there are more good than any threshold
    if len(matched_keypoints) > 20:
        out = cv.drawMatchesKnn(
            source_img,
            source["keypoint"],
            target_img,
            target_kp,
            matched_keypoints,
            None,
            flags=2,
        )

Below is the entire source code.

get_features_from_file.py


import cv2 as cv
import pickle


def get_sources():
    """Get source's features from file

        Returns:
            sources(list): source's keypoints, descriptors,and img
        """
    #Get feature point information from a file
    with open("sources_data.pickle", mode="rb") as f:
        sources = pickle.load(f)

    for n in sources:
        items = sources[n]
        #Change keypoints from bytes to list
        list(map(list, items["keypoint"]))
        #Restore keypoints to original structure
        keypoints = []
        for p in items["keypoint"]:
            temp = cv.KeyPoint(
                x=p[0][0],
                y=p[0][1],
                _size=p[1],
                _angle=p[2],
                _response=p[3],
                _octave=p[4],
                _class_id=p[5],
            )
            keypoints.append(temp)
        items["keypoint"] = keypoints

    return sources


matcher = cv.BFMatcher()

#Load target image
target_img = cv.imread("images/target_girl.jpg ")
#Feature acquisition
akaze = cv.AKAZE_create()
target_kp, target_des = akaze.detectAndCompute(target_img, None)
#Read the feature point information of the source image from the file
sources = get_sources()

for n in sources:
    source = sources[n]
    source_img = cv.imread("images/" + source["src"])
    matches = matcher.knnMatch(source["descriptor"], target_des, k=2)
    #Thin out data
    ratio = 0.5
    matched_keypoints = []
    for m, n in matches:
        if m.distance < ratio * n.distance:
            matched_keypoints.append([m])

    #Output the result image when there are more good than any threshold
    if len(matched_keypoints) > 20:
        out = cv.drawMatchesKnn(
            source_img,
            source["keypoint"],
            target_img,
            target_kp,
            matched_keypoints,
            None,
            flags=2,
        )

cv.imwrite("images/result.jpg ", out)
cv.waitKey()

result

In the result image below, the feature points matched between the source image and the target image are drawn. I was able to find the target girl safely! image.png

At the end

The key to saving feature points in a file is Accessing cv :: KeyPoint Make a list based on the accessed information is. If you have a chance to process images with OpenCV, please give it a try.

Reference page

Export KeyPoint of OpenCV for Python3 to a file

[Feature point matching with python3, opencv3 (AKAZE, KNN)] (https://techtech-sorae.com/python3opencv3%E3%81%A7%E7%89%B9%E5%BE%B4%E7%82%B9%E3%83%9E%E3%83%83%E3%83%81%E3%83%B3%E3%82%B0akaze-knn/)

Recommended Posts

How to save the feature point information of an image in a file and use it for matching
Extract the lyrics information in the MP3 / MP4 file and save it in the lyrics file (* .lrc) for Sony walkman.
[Introduction to Python] How to use the in operator in a for statement?
How to get the vertex coordinates of a feature in ArcPy
Start the webcam to take a still image and save it locally
Use Cloud Dataflow to dynamically change the destination according to the value of the data and save it in GCS
The story of making a tool to load an image with Python ⇒ save it as another name
[Python] How to save the installed package and install it in a new environment at once Mac environment
How to count the number of elements in Django and output to a template
Use Pillow to make the image transparent and overlay only part of it
Save an array of numpy to a wav file using the wave module
How to read a serial number file in a loop, process it, and graph it
How to use Decorator in Django and how to make it
How to display the modification date of a file in C language up to nanoseconds
Save the pystan model and results in a pickle file
How to use the __call__ method in a Python class
Comparison of how to use higher-order functions in Python 2 and 3
How to interactively draw a machine learning pipeline with scikit-learn and save it in HTML
I made a function to crop the image of python openCV, so please use it.
I tried to find the affine matrix in image alignment (feature point matching) using affine transformation.
An easy way to view the time taken in Python and a smarter way to improve it
It may be a problem to use Japanese for folder names and notebook names in Databricks
How to determine the existence of a selenium element in Python
How to know the internal structure of an object in Python
How to check the memory size of a variable in Python
How to check the memory size of a dictionary in Python
How to output the output result of the Linux man command to a file
How to use machine learning for work? 01_ Understand the purpose of machine learning
How to take a screenshot of the Chrome screen (prevent it from cutting off in the middle)
[Python] What is a slice? An easy-to-understand explanation of how to use it with a concrete example.
How to get the pixel value of the point from the satellite image by specifying the latitude and longitude
[Python] The role of the asterisk in front of the variable. Divide the input value and assign it to a variable
[Python] How to use the for statement. A method of extracting by specifying a range or conditions.
Learn the flow of Bayesian estimation and how to use Pystan through a simple regression model
I want to extract the tag information (title and artist) of a music file (flac, wav).
Started Python: Swap an array of values obtained from SQL results to a list type and use it in IN of another query
From the introduction of GoogleCloudPlatform Natural Language API to how to use it
How to split and save a DataFrame
[Python / Jupyter] Translate the comment of the program copied to the clipboard and insert it in a new cell
I made a POST script to create an issue on Github and register it in the Project
How to send a visualization image of data created in Python to Typetalk
How to specify a .ui file in the dialog / widget GUI in PySide
How to put a line number at the beginning of a CSV file
How to make a container name a subdomain and make it accessible in Docker
Open an Excel file in Python and color the map of Japan
I made a program to check the size of a file in Python
How to copy and paste the contents of a sheet in Google Spreadsheet in JSON format (using Google Colab)
[python] Send the image captured from the webcam to the server and save it
How to use is and == in Python
How to input a character string in Python and output it as it is or in the opposite direction.
The tree.plot_tree of scikit-learn was very easy and convenient, so I tried to summarize how to use it easily.
An engineer who has noticed the emo of cryptography is trying to implement it in Python and defeat it
GAE --With Python, rotate the image based on the rotation information of EXIF and upload it to Cloud Storage.
I thought it would be slow to use a for statement in NumPy, but that wasn't the case.
Various ways to read the last line of a csv file in Python
How to pass the execution result of a shell command in a list in Python
Create a Python image in Django without a dummy image file and test the image upload
How to mention a user group in slack notification, how to check the id of the user group
Use libsixel to output Sixel in Python and output a Matplotlib graph to the terminal.
I want to write an element to a file with numpy and check it.
[Python] Concatenate a List containing numbers and write it to an output file.