I created an image cropping tool using OpenCV. Cut out the object, project it, shape it, and save each one.
The object is assumed to be a film with the following image. Even if this is not the case, I think it can be used when extracting rectangles.
<img width="200", alt="sample.jpg ", src="https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/779817/f532ad11-9c6d-e7df-844b-a30e6a2851ea.jpeg "> <img width="100", alt="sample2.png ", src="https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/779817/528427a2-9619-e2d1-b189-8a513cf87c64.jpeg ">
Also, since I stumbled on the projective transformation during creation, the content and solution are described at the end. Details
Mac OS python 3.8.5
opencv-python 4.4.0.44 numpy 1.19.2 tqdm 4.50.2
python
pip install opencv-python
pip install tqdm
I am importing tqdm to use the progress bar.
It is a process to correct the object as if it was taken from the front. I used it because I wanted the cropped image to be at a right angle.
<img width="350", alt="射影変換後.jpeg ", src="https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/779817/5b69d421-8b17-d09c-b6d4-e7843473ccb2.jpeg ">
Reference article for projective transformation Image correction is easy with Python / OpenCV projective transformation! | WATLAB -Python, signal processing, AI- Try projective transformation of images using OpenCV with Python --Qiita
I refer to the following articles to make Japanese paths compatible with image reading. About dealing with problems when handling file paths including Japanese in Python OpenCV cv2.imread and cv2.imwrite --Qiita
The operation method is
The result will be saved as "./result folder / image file name / image file name_0, ...". If the same folder as the file name exists in result, the process is skipped.
If you can't cut it well, try changing thresh_value or minimum_area. [Image Threshold Processing — OpenCV-Python Tutorials 1 documentation](http://labs.eecs.tottori-u.ac.jp/sd/Member/oyamada/OpenCV/html/py_tutorials/py_imgproc/py_thresholding/py_thresholding. html) [Area (contour) features — OpenCV-Python Tutorials 1 documentation](http://labs.eecs.tottori-u.ac.jp/sd/Member/oyamada/OpenCV/html/py_tutorials/py_imgproc/py_contours/py_contour_features/ py_contour_features.html)
python
import os, shutil, time
from pathlib import Path
import cv2
import numpy as np
from tqdm import tqdm
thresh_value = 240 #Boundary value when binarizing,If the pixel value is smaller than this, make it white(max:255)
minimum_area = 10000 #Do not process objects smaller than this when the contour is acquired(For when dots other than the target object are detected)
def imread(filename, flags=cv2.IMREAD_COLOR, dtype=np.uint8):
try:
n = np.fromfile(filename, dtype)
img = cv2.imdecode(n, flags)
return img
except Exception as e:
print(e)
return None
def imwrite(filename, img, params=None):
try:
ext = os.path.splitext(filename)[1]
result, n = cv2.imencode(ext, img, params)
if result:
with open(filename, mode='w+b') as f:
n.tofile(f)
return True
else:
return False
except Exception as e:
print(e)
return False
def calculate_width_height(pts, add):
"""
Width of detected shape,Find the height using three squares
If it is too slanted, the shape will change when the projective transformation is performed.
:parameter
------------
pts: numpy.ndarray
Coordinates of 4 points of the extracted shape, shape=(4, 1, 2)
add: int
Correction because the coordinates of the start point differ depending on the shape
:return
------------
width: int
Calculated width
height: int
Calculated height
"""
top_left_cood = pts[0 + add][0]
bottom_left_cood = pts[1 + add][0]
bottom_right_cood = pts[2 + add][0]
width = np.int(np.linalg.norm(bottom_left_cood - bottom_right_cood))
height = np.int(np.linalg.norm(top_left_cood - bottom_left_cood))
return width, height
def img_cut():
"""
Images in the resource folder(jpg, png)To get the outline of the object
Cut the object, project it, and bring it to the front
1.folder,Read file
2.Image reading,Binarization(Black and white)processing
3.Get contour
4.Homography
5.output
6.Move resource file to result
:return: None
"""
# 1.folder,Read file
resource_folder = Path(r'./resource')
result_folder = Path(r'./result')
#Create if result folder does not exist
if not result_folder.exists():
result_folder.mkdir()
img_list1 = list(resource_folder.glob('*.jpg')) #Path list of jpg files in the folder
img_list2 = list(resource_folder.glob('*.jpeg'))
img_list3 = list(resource_folder.glob('*.png'))
img_list = img_list1 + img_list2 + img_list3
for img in img_list:
img_name, img_suffix = img.stem, img.suffix #Get image name and extension
#Create a folder with the image file name in the result folder,Skip conversion if the same folder already exists
result_img_folder = Path(r'./result/{}'.format(img_name))
if not result_img_folder.exists():
result_img_folder.mkdir()
else:
print('{}Cannot be converted because a folder with the same name as exists in result'.format(img_name))
continue
# 2.Image reading,Binarization(Black and white)processing
read_img = imread(str(img))
gray_img = cv2.cvtColor(read_img, cv2.COLOR_BGR2GRAY)
ret, thresh_img = cv2.threshold(gray_img, thresh_value, 255, cv2.THRESH_BINARY_INV)
# --------------------------------------------
#For binarized image confirmation
# cv2.namedWindow('final', cv2.WINDOW_NORMAL)
# cv2.imshow('final', thresh_img)
# cv2.waitKey(0)
# cv2.destroyAllWindows()
# --------------------------------------------
# 3.Get contour
# cv2.RETR_EXTERNAL:Extract only the outermost contour from the detected contours->Ignore contours even if they are inside the contour
# cv2.CHAIN_APPROX_SIMPLE:Get only 4 corners, not the edges of the contour
contours, hierarchy = cv2.findContours(thresh_img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
process_cnt = [] #List of contours to actually cut
for cnt in contours:
if cv2.contourArea(cnt) < minimum_area: #Do not cut objects whose area in the contour is too small
continue
process_cnt.append(cnt)
num = 0
for p_cnt in tqdm(process_cnt[::-1], desc='{}'.format(img_name)): #For some reason, the process starts from the bottom image, so slice it in reverse.(Up)Fix from
x, y, w, h = cv2.boundingRect(p_cnt) #Top left of contour x,y coordinate&width,Get height
img_half_width = x + w / 2
# cv2.arcLength:Peripheral length of contour,True means the contour is closed
# cv2.approPolyDP:Approximation of detected shape
epsilon = 0.1 * cv2.arcLength(p_cnt, True)
approx = cv2.approxPolyDP(p_cnt, epsilon, True)
try:
# 4.Homography
pts1 = np.float32(approx)
if pts1[0][0][0] < img_half_width: #If the start point of the coordinates stored in pts is the upper left
width, height = calculate_width_height(pts1, 0)
pts2 = np.float32([[0, 0], [0, height], [width, height], [width, 0]])
else:
width, height = calculate_width_height(pts1, 1)
pts2 = np.float32([[width, 0], [0, 0], [0, height], [width, height]])
except IndexError:
continue
M = cv2.getPerspectiveTransform(pts1, pts2)
dst = cv2.warpPerspective(read_img, M, (width, height))
result_img_name = img_name + '_{}.{}'.format(num, img_suffix)
imwrite(str(result_img_folder) + '/' + result_img_name, dst)
num += 1
# 6.Move resource file to result
shutil.move(str(img), result_img_folder)
if __name__ == '__main__':
img_cut()
print('End of execution')
time.sleep(3)
python
# cv2.arcLength:Peripheral length of contour,True means the contour is closed
# cv2.approPolyDP:Approximation of detected shape
epsilon = 0.1 * cv2.arcLength(p_cnt, True)
approx = cv2.approxPolyDP(p_cnt, epsilon, True)
try:
# 4.Homography
pts1 = np.float32(approx)
The coordinate information of the four corners of the object is stored in pts1. ex) [[[6181. 598.]]
[[ 145. 656.]]
[[ 135. 3499.]]
[[6210. 3363.]]]
By bringing these four points to the corners of the image, the image looks like it was seen from the front.
python
if pts1[0][0][0] < img_half_width: #If the start point of the coordinates stored in pts is the upper left
width, height = calculate_width_height(pts1, 0)
pts2 = np.float32([[0, 0], [0, height], [width, height], [width, 0]])
else:
width, height = calculate_width_height(pts1, 1)
pts2 = np.float32([[width, 0], [0, 0], [0, height], [width, height]])
It determines where the starting point of pts1 is. Since the stored 4 coordinate points are stored counterclockwise starting from the top point (the y-axis is small), the start point of pts1 changes depending on the tilt of the image. It is judged and made to correspond to the coordinates pts2 after the projective transformation. <img width="400", alt="pts座標.png ", src="https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/779817/f6c5e9a7-d49c-45ca-191e-05efa952f14a.png ">
The judgment method is based on whether the x-coordinate of the start point is to the left or right of the center of the image.
Also, since I want to keep the shape of the object as much as possible, I calculated the width and height using three squares and output a cropped image with that size.
I didn't know that the starting point of pts1 would change, and at first I was outputting a meaningless image. I couldn't find it easily even if I searched on the net, and at the end I was in a state where I finally found out by glaring at the image. I hope it will be helpful when there are people who are similarly in trouble.
tutorial [Image Threshold Processing — OpenCV-Python Tutorials 1 documentation](http://labs.eecs.tottori-u.ac.jp/sd/Member/oyamada/OpenCV/html/py_tutorials/py_imgproc/py_thresholding/py_thresholding. html) Outline: First Step — OpenCV-Python Tutorials 1 documentation [Area (contour) features — OpenCV-Python Tutorials 1 documentation](http://labs.eecs.tottori-u.ac.jp/sd/Member/oyamada/OpenCV/html/py_tutorials/py_imgproc/py_contours/py_contour_features/ py_contour_features.html)
OpenCV reference (contour, image cropping, projective transformation) Basics of contour detection from images with OpenCV (per findContours function) | North Building Tech.com [Get and crop objects in the image using contours-python, image, opencv-contour](https://living-sun.com/ja/python/725302-getting-and-cropping-object-in] -images-using-contours-python-image-opencv-contour.html) Try projective transformation of images using OpenCV with Python --Qiita
OpenCV Japanese pass reading support About dealing with problems when handling file paths including Japanese in Python OpenCV cv2.imread and cv2.imwrite --Qiita
numpy three square calculation reference [Calculate Euclidean distance](http://harmonizedai.com/article/%E3%83%A6%E3%83%BC%E3%82%AF%E3%83%AA%E3%83%83%E3% 83% 89% E8% B7% 9D% E9% 9B% A2% E3% 82% 92% E6% B1% 82% E3% 82% 81% E3% 82% 8B /)
Progress bar reference Display progress bar with tqdm --Qiita
Recommended Posts