Image cropping tool creation with OpenCV, precautions for projective conversion (only a little)

Introduction

I created an image cropping tool using OpenCV. Cut out the object, project it, shape it, and save each one.

The object is assumed to be a film with the following image. Even if this is not the case, I think it can be used when extracting rectangles.

<img width="200", alt="sample.jpg ", src="https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/779817/f532ad11-9c6d-e7df-844b-a30e6a2851ea.jpeg ">       <img width="100", alt="sample2.png ", src="https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/779817/528427a2-9619-e2d1-b189-8a513cf87c64.jpeg ">

Also, since I stumbled on the projective transformation during creation, the content and solution are described at the end. Details

environment

Mac OS python 3.8.5

opencv-python 4.4.0.44 numpy 1.19.2 tqdm 4.50.2

python


pip install opencv-python
pip install tqdm

I am importing tqdm to use the progress bar.

Homography

It is a process to correct the object as if it was taken from the front. I used it because I wanted the cropped image to be at a right angle.

<img width="350", alt="射影変換後.jpeg ", src="https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/779817/5b69d421-8b17-d09c-b6d4-e7843473ccb2.jpeg ">

Reference article for projective transformation Image correction is easy with Python / OpenCV projective transformation! | WATLAB -Python, signal processing, AI- Try projective transformation of images using OpenCV with Python --Qiita

Full text

I refer to the following articles to make Japanese paths compatible with image reading. About dealing with problems when handling file paths including Japanese in Python OpenCV cv2.imread and cv2.imwrite --Qiita

The operation method is

  1. Create a resource folder in the same hierarchy as the script
  2. Put the image you want to process in the resource folder (multiple images allowed, jpg, jpeg or png)
  3. Run

The result will be saved as "./result folder / image file name / image file name_0, ...". If the same folder as the file name exists in result, the process is skipped.

If you can't cut it well, try changing thresh_value or minimum_area. [Image Threshold Processing — OpenCV-Python Tutorials 1 documentation](http://labs.eecs.tottori-u.ac.jp/sd/Member/oyamada/OpenCV/html/py_tutorials/py_imgproc/py_thresholding/py_thresholding. html) [Area (contour) features — OpenCV-Python Tutorials 1 documentation](http://labs.eecs.tottori-u.ac.jp/sd/Member/oyamada/OpenCV/html/py_tutorials/py_imgproc/py_contours/py_contour_features/ py_contour_features.html)

python


import os, shutil, time
from pathlib import Path
import cv2
import numpy as np
from tqdm import tqdm

thresh_value = 240  #Boundary value when binarizing,If the pixel value is smaller than this, make it white(max:255)
minimum_area = 10000  #Do not process objects smaller than this when the contour is acquired(For when dots other than the target object are detected)


def imread(filename, flags=cv2.IMREAD_COLOR, dtype=np.uint8):
    try:
        n = np.fromfile(filename, dtype)
        img = cv2.imdecode(n, flags)
        return img
    except Exception as e:
        print(e)
        return None


def imwrite(filename, img, params=None):
    try:
        ext = os.path.splitext(filename)[1]
        result, n = cv2.imencode(ext, img, params)

        if result:
            with open(filename, mode='w+b') as f:
                n.tofile(f)
            return True
        else:
            return False
    except Exception as e:
        print(e)
        return False


def calculate_width_height(pts, add):
    """
Width of detected shape,Find the height using three squares
If it is too slanted, the shape will change when the projective transformation is performed.

    :parameter
    ------------
    pts: numpy.ndarray
Coordinates of 4 points of the extracted shape, shape=(4, 1, 2)
    add: int
Correction because the coordinates of the start point differ depending on the shape

    :return
    ------------
    width: int
Calculated width
    height: int
Calculated height
    """
    top_left_cood = pts[0 + add][0]
    bottom_left_cood = pts[1 + add][0]
    bottom_right_cood = pts[2 + add][0]

    width = np.int(np.linalg.norm(bottom_left_cood - bottom_right_cood))
    height = np.int(np.linalg.norm(top_left_cood - bottom_left_cood))

    return width, height


def img_cut():
    """
Images in the resource folder(jpg, png)To get the outline of the object
Cut the object, project it, and bring it to the front

    1.folder,Read file
    2.Image reading,Binarization(Black and white)processing
    3.Get contour
    4.Homography
    5.output
    6.Move resource file to result

    :return: None
    """

    # 1.folder,Read file
    resource_folder = Path(r'./resource')
    result_folder = Path(r'./result')
    #Create if result folder does not exist
    if not result_folder.exists():
        result_folder.mkdir()

    img_list1 = list(resource_folder.glob('*.jpg'))  #Path list of jpg files in the folder
    img_list2 = list(resource_folder.glob('*.jpeg'))
    img_list3 = list(resource_folder.glob('*.png'))
    img_list = img_list1 + img_list2 + img_list3

    for img in img_list:
        img_name, img_suffix = img.stem, img.suffix  #Get image name and extension

        #Create a folder with the image file name in the result folder,Skip conversion if the same folder already exists
        result_img_folder = Path(r'./result/{}'.format(img_name))
        if not result_img_folder.exists():
            result_img_folder.mkdir()
        else:
            print('{}Cannot be converted because a folder with the same name as exists in result'.format(img_name))
            continue

        # 2.Image reading,Binarization(Black and white)processing
        read_img = imread(str(img))
        gray_img = cv2.cvtColor(read_img, cv2.COLOR_BGR2GRAY)
        ret, thresh_img = cv2.threshold(gray_img, thresh_value, 255, cv2.THRESH_BINARY_INV)

        # --------------------------------------------
        #For binarized image confirmation
        # cv2.namedWindow('final', cv2.WINDOW_NORMAL)
        # cv2.imshow('final', thresh_img)
        # cv2.waitKey(0)
        # cv2.destroyAllWindows()
        # --------------------------------------------

        # 3.Get contour
        # cv2.RETR_EXTERNAL:Extract only the outermost contour from the detected contours->Ignore contours even if they are inside the contour
        # cv2.CHAIN_APPROX_SIMPLE:Get only 4 corners, not the edges of the contour
        contours, hierarchy = cv2.findContours(thresh_img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
        process_cnt = []  #List of contours to actually cut
        for cnt in contours:
            if cv2.contourArea(cnt) < minimum_area:  #Do not cut objects whose area in the contour is too small
                continue
            process_cnt.append(cnt)

        num = 0
        for p_cnt in tqdm(process_cnt[::-1], desc='{}'.format(img_name)):  #For some reason, the process starts from the bottom image, so slice it in reverse.(Up)Fix from
            x, y, w, h = cv2.boundingRect(p_cnt)  #Top left of contour x,y coordinate&width,Get height
            img_half_width = x + w / 2

            # cv2.arcLength:Peripheral length of contour,True means the contour is closed
            # cv2.approPolyDP:Approximation of detected shape
            epsilon = 0.1 * cv2.arcLength(p_cnt, True)
            approx = cv2.approxPolyDP(p_cnt, epsilon, True)
            try:
                # 4.Homography
                pts1 = np.float32(approx)
                if pts1[0][0][0] < img_half_width:  #If the start point of the coordinates stored in pts is the upper left
                    width, height = calculate_width_height(pts1, 0)
                    pts2 = np.float32([[0, 0], [0, height], [width, height], [width, 0]])
                else:
                    width, height = calculate_width_height(pts1, 1)
                    pts2 = np.float32([[width, 0], [0, 0], [0, height], [width, height]])
            except IndexError:
                continue
            M = cv2.getPerspectiveTransform(pts1, pts2)
            dst = cv2.warpPerspective(read_img, M, (width, height))

            result_img_name = img_name + '_{}.{}'.format(num, img_suffix)
            imwrite(str(result_img_folder) + '/' + result_img_name, dst)

            num += 1
        # 6.Move resource file to result
        shutil.move(str(img), result_img_folder)


if __name__ == '__main__':
    img_cut()
    print('End of execution')
    time.sleep(3)

Details

python


# cv2.arcLength:Peripheral length of contour,True means the contour is closed
# cv2.approPolyDP:Approximation of detected shape
epsilon = 0.1 * cv2.arcLength(p_cnt, True)
approx = cv2.approxPolyDP(p_cnt, epsilon, True)
try:
    # 4.Homography
    pts1 = np.float32(approx)

The coordinate information of the four corners of the object is stored in pts1. ex) [[[6181. 598.]]

[[ 145. 656.]]

[[ 135. 3499.]]

[[6210. 3363.]]]

By bringing these four points to the corners of the image, the image looks like it was seen from the front.

python


if pts1[0][0][0] < img_half_width:  #If the start point of the coordinates stored in pts is the upper left
    width, height = calculate_width_height(pts1, 0)
    pts2 = np.float32([[0, 0], [0, height], [width, height], [width, 0]])
else:
    width, height = calculate_width_height(pts1, 1)
    pts2 = np.float32([[width, 0], [0, 0], [0, height], [width, height]])

It determines where the starting point of pts1 is. Since the stored 4 coordinate points are stored counterclockwise starting from the top point (the y-axis is small), the start point of pts1 changes depending on the tilt of the image. It is judged and made to correspond to the coordinates pts2 after the projective transformation. <img width="400", alt="pts座標.png ", src="https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/779817/f6c5e9a7-d49c-45ca-191e-05efa952f14a.png ">

The judgment method is based on whether the x-coordinate of the start point is to the left or right of the center of the image.

Also, since I want to keep the shape of the object as much as possible, I calculated the width and height using three squares and output a cropped image with that size.

At the end

I didn't know that the starting point of pts1 would change, and at first I was outputting a meaningless image. I couldn't find it easily even if I searched on the net, and at the end I was in a state where I finally found out by glaring at the image. I hope it will be helpful when there are people who are similarly in trouble.

Reference site

tutorial [Image Threshold Processing — OpenCV-Python Tutorials 1 documentation](http://labs.eecs.tottori-u.ac.jp/sd/Member/oyamada/OpenCV/html/py_tutorials/py_imgproc/py_thresholding/py_thresholding. html) Outline: First Step — OpenCV-Python Tutorials 1 documentation [Area (contour) features — OpenCV-Python Tutorials 1 documentation](http://labs.eecs.tottori-u.ac.jp/sd/Member/oyamada/OpenCV/html/py_tutorials/py_imgproc/py_contours/py_contour_features/ py_contour_features.html)

OpenCV reference (contour, image cropping, projective transformation) Basics of contour detection from images with OpenCV (per findContours function) | North Building Tech.com [Get and crop objects in the image using contours-python, image, opencv-contour](https://living-sun.com/ja/python/725302-getting-and-cropping-object-in] -images-using-contours-python-image-opencv-contour.html) Try projective transformation of images using OpenCV with Python --Qiita

OpenCV Japanese pass reading support About dealing with problems when handling file paths including Japanese in Python OpenCV cv2.imread and cv2.imwrite --Qiita

numpy three square calculation reference [Calculate Euclidean distance](http://harmonizedai.com/article/%E3%83%A6%E3%83%BC%E3%82%AF%E3%83%AA%E3%83%83%E3% 83% 89% E8% B7% 9D% E9% 9B% A2% E3% 82% 92% E6% B1% 82% E3% 82% 81% E3% 82% 8B /)

Progress bar reference Display progress bar with tqdm --Qiita

Recommended Posts

Image cropping tool creation with OpenCV, precautions for projective conversion (only a little)
Image processing with Lambda + OpenCV (gray image creation)
GUI image cropping tool made with Python + Tkinter
Problems when creating a csv-json conversion tool with python
Create a simple video analysis tool with python wxpython + openCV
[Python] Accessing and cropping image pixels using OpenCV (for beginners)