I tried to sort out the objects from the image of the steak set meal-② Overlap number sorting

Introduction

Last time, Selective Search was used to detect objects in the image of the steak set meal. Although the accuracy has improved, I noticed that out of about 50 images, they are classified into about 3 types: dust-like images, the same images (with slightly different cropping positions), and necessary object images. .. This time, I will try to see if I can select only the necessary object images from them.

hypothesis

Isn't an image with a large number of overlaps necessary when extracting an object from an image? However, in reality, it is possible to exclude the rectangular child part of the parent-child relationship.

What is this guy saying

When detecting an object from an image, it is extracted as a rectangle, but the part where the rectangles are mixed is called the overlap. The red part of the image below.

overlap.png

Reference source

-(Fun hit detection course 1-Hit judgment between rectangles (beginner)-) [http://d.hatena.ne.jp/ono36/20070718/p1]

Source code

I wrote it for the time being.

group_image


# -*- coding: utf-8 -*-

import cv2
import matplotlib.pyplot as plt
import matplotlib.patches as mpatches
import selectivesearch
import os

def main():
    # loading lena image
    img = cv2.imread("{Steak set meal image}")

    # perform selective search
    img_lbl, regions = selectivesearch.selective_search(
        img,
        scale=500,
        sigma=0.9,
        min_size=10
    )

    candidates = set()

    for r in regions:
        # excluding same rectangle (with different segments)
        if r['rect'] in candidates:
            continue

        # excluding regions smaller than 2000 pixels
        if r['size'] < 2000:
            continue

        # distorted rects
        x, y, w, h = r['rect']

        if w / h > 1.2 or h / w > 1.2:
            continue

        candidates.add(r['rect'])

    # draw rectangles on the original image
    fig, ax = plt.subplots(ncols=1, nrows=1, figsize=(6, 6))
    ax.imshow(img)

    overlaps = {}

    #Count the number of overlaps and assign them to the array.
    for x, y, w, h in candidates:
        group = '%s_%s_%s_%s' % (x, y, w, h)

        for x2, y2, w2, h2 in candidates:
            if x2 - w < x < x2 + w2 and y2 - h < y < y2 + h2:

                if not group in overlaps:
                    overlaps[group] = 0

                overlaps[group] = overlaps[group] + 1

    print overlaps

    #Outputs files with 30 or more overlaps (30 is arbitrarily thresholded).
    for key, overlap in enumerate(overlaps):
        if overlap > 30:
            for x, y, w, h in candidates:
                group = x + y + w + h

                if group in overlaps:
                    cv2.imwrite("{Directory path}" + str(group) + '.jpg', img[y:y + h, x:x + w])

Supplement

--Object Determines if it overlaps with the detected image.

if x2 - w < x < x2 + w2 and y2 - h < y < y2 + h2:

--Only the extraction results with 30 or more overlaps are saved as images.

result

(Original) 50 sheets → 36 sheets

About 30% of the images of the steak set meal have been removed.

In addition, 5 types of object images of previous remained.

Summary

――This time, since there is only one type of verification image, we have made some adjustments and result judgments, so if you try with other images, you may get different results. ――Next time, I would like to do something like clustering.

All page links

-I tried object detection using Python and OpenCV -I tried to sort out objects from the image of steak set meal-① Object detection -I tried to sort out the objects from the image of the steak set meal-② Overlap number sorting -I tried to sort out the objects from the image of the steak set meal-③ Similar image heat map detection -I tried to sort out the objects from the image of the steak set meal-④ Clustering -I tried to sort out objects from the image of steak set meal-⑤ Similar image feature point detection edition

Recommended Posts

I tried to sort out the objects from the image of the steak set meal-② Overlap number sorting
I tried to sort out the objects from the image of the steak set meal-④ Clustering
I tried to sort out the objects from the image of the steak set meal --③ Similar image Heat map detection
I tried to sort out the objects from the image of the steak set meal-⑤ Similar image feature point detection
I tried to cut out a still image from the video
I tried to correct the keystone of the image
I tried to detect the iris from the camera image
I tried to predict the number of people infected with coronavirus in consideration of the effect of refraining from going out
I tried to find the entropy of the image with python
I tried to find the trend of the number of ships in Tokyo Bay from satellite images.
I tried to build the SD boot image of LicheePi Nano
I tried to transform the face image using sparse_image_warp of TensorFlow Addons
I tried to get the batting results of Hachinai using image processing
(Python) I tried to analyze 1 million hands ~ I tried to estimate the number of AA ~
I tried to touch the API of ebay
I tried using the image filter of OpenCV
I tried to predict the price of ETF
I tried to vectorize the lyrics of Hinatazaka46!
[Python] I tried to judge the member image of the idol group using Keras
I tried to automate the face hiding work of the coordination image for wear
I tried to summarize the basic form of GPLVM
I tried to put out the frequent word ranking of LINE talk with Python
I tried to extract the text in the image file using Tesseract of the OCR engine
I tried to visualize the spacha information of VTuber
I tried to erase the negative part of Meros
I tried to classify the voices of voice actors
I tried to compress the image using machine learning
I tried to summarize the string operations of Python
I tried to tabulate the number of deaths per capita of COVID-19 (new coronavirus) by country
I tried to predict the genre of music from the song title on the Recurrent Neural Network
Implementation of recommendation system ~ I tried to find the similarity from the outline of the movie using TF-IDF ~
I tried using PI Fu to generate a 3D model of a person from one image
[Horse Racing] I tried to quantify the strength of racehorses
I tried "gamma correction" of the image with Python + OpenCV
I tried to get the location information of Odakyu Bus
I tried to find the average of the sequence with TensorFlow
[Python] I tried to visualize the follow relationship of Twitter
[Machine learning] I tried to summarize the theory of Adaboost
I tried to fight the Local Minimum of Goldstein-Price Function
I tried changing the python script from 2.7.11 to 3.6.0 on windows10
I tried to get various information from the codeforces API
I tried to predict the number of domestically infected people of the new corona with a mathematical model
I tried fitting the exponential function and logistics function to the number of COVID-19 positive patients in Tokyo
[Linux] I tried to summarize the command of resource confirmation system
I tried to get the index of the list using the enumerate function
I tried to automate the watering of the planter with Raspberry Pi
[Python] Try to graph from the image of Ring Fit [OCR]
I tried to process the image in "sketch style" with OpenCV
I tried to process the image in "pencil style" with OpenCV
I tried to expand the size of the logical volume with LVM
I tried to summarize the frequently used implementation method of pytest-mock
I tried to improve the efficiency of daily work with Python
Find all patterns to extract a specific number from the set
I tried to visualize the common condition of VTuber channel viewers
[Python] I tried to reproduce the emergency escape program to return from the world to return from the modified world of "The disappearance of Haruhi Suzumiya"
I tried to make a thumbnail image of the best avoidance flag-chan! With RGB values ​​[Histogram] [Visualization]
[Completed version] Try to find out the number of residents in the town from the address list with Python
I tried to move the ball
I tried to deliver mail from Node.js and Python using the mail delivery service (SendGrid) of IBM Cloud!
I tried to estimate the interval.
[Python] I tried to summarize the set type (set) in an easy-to-understand manner.