I made a program to solve (hint) Saizeriya's spot the difference

Hello. This is the first post. Waiting time when I went to Saizeriya the other day About 20 minutes I searched for mistakes in Saizeriya and could not find it ... (too much ...) So, I thought that it could be solved by image processing, so I tried it. It was also a good study for OpenCV.

Thing you want to do

Image data officially released by Saizeriya using the OpenCV library (https://www.saizeriya.co.jp/entertainment/) I want to process and automate the search for mistakes! !! !! What you actually do is

--Process the image (remove margins, split in half) --Calculate image differences --Display difference information on the original image

It's like that. I gave the code to GitHub.

Execution environment

macOS Mojave 10.14.4 Python 3.6.7 OpenCV 3.4.1

Process the image

When you download it, you will notice that "the images to be compared are stuck together" + "there is a mysterious margin".

• (It's hard to understand because the surroundings are also white ...) Quoted from Saizeriya's site *

First of all, I would like to delete the margins. The flow of deleting blanks is --Make a grayscale image ―― Binarize --Extract the contour --Get and cut x and y from the contours that have the smallest and largest coordinates.

[https://qiita.com/trami/items/e25eb70a59a51ae4f7ba# How to remove margins](https://qiita.com/trami/items/e25eb70a59a51ae4f7ba#%E3%81%A9%E3%81 % AE% E3% 82% 88% E3% 81% 86% E3% 81% AB% E3% 81% 97% E3% 81% A6% E4% BD% 99% E7% 99% BD% E5% 89% 8A We refer to the article% E9% 99% A4% E3% 82% 92% E8% A1% 8C% E3% 81% 86% E3% 81% AE% E3% 81% 8B).

The grayscale function in OpenCV

gray.py

gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY) #Grayscale

The binarization function

binary.py

r, binary = cv2.threshold(gray, 0, 255,cv2.THRESH_OTSU)  #Binarization

The image cropping function

cut_img.py

img = img[y1:y2,x1:x2] #(x1,y1)From(x2,y2)Cut out

It's like that. If you combine these, the code so far will be

saizeriya.py

import cv2
import numpy as np

img = cv2.imread('diff.png') #Loading images
height, width, d = img.shape #Height, width, depth
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY) #Grayscale
r, binary = cv2.threshold(gray, 0, 255,cv2.THRESH_OTSU)  #Binar at threshold 200
contours = cv2.findContours(binary, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)[0]

#Extract contours, remove margins
x1 = [] #Minimum x coordinate
y1 = [] #Minimum y coordinate
x2 = [] #Maximum x coordinate
y2 = [] #Maximum y coordinate
for i in range(1, len(contours)):
    ret = cv2.boundingRect(contours[i])
    x1.append(ret[0])
    y1.append(ret[1])
    x2.append(ret[0] + ret[2])
    y2.append(ret[1] + ret[3])
x1_min = min(x1)
y1_min = min(y1)
x2_max = max(x2)
y2_max = max(y2)
img = img[0:600, x1_min:x2_max] #Cut off the margin

The execution result is as follows. You can see that it is cut well.

Then cut the image in half.

saizeriya.py

height, width,d= img.shape #Height, width, depth(The size of the cut out)
midle = int(width/2)

img1 = img[0:height,0:midle-3]
img2 = img[0:height,midle-8:width-11] #Actually, the width is a little different... 

What was ridiculous here was that the sharp edges of the two images were actually slightly different. After all, I adjusted the part to be cut manually.

Calculate image differences

Then calculate the difference between the images. The difference between the images can be subtracted between the numpy arrays, but when you actually execute it, the difference is difficult to understand (a slight difference in color and a slight deviation in location will be displayed as a difference), so this time OpenCV I used the ** absdiff ** function of.

The ** absdiff ** function can find the absolute value of the difference between two images.

saizeriya.py

#Show diff
result = np.copy(img1) #Array to store the resulting image
add = np.copy(img1) #Array to store the resulting image
#result = img1-img2 #Calculation of difference
result = cv2.absdiff(img1, img2) #Calculation of difference(absdiff)

Execution result

For reference, it is the result when it is executed by subtraction between arrays.

• I don't know what's different about this ... *

Display the difference information on the original image

With this, I don't know where the difference is, so I would like to combine it with the original image. The original gray image + difference image (color) is combined.

The function to use is the ** add ** function. Also, use the ** cvtCOLOR ** function to convert the original color image.

Color 3D array → Gray 2D array → Gray 3D array

Process with.

saizeriya.py

img3 = cv2.cvtColor(img2,cv2.COLOR_BGR2GRAY) #Grayscale
img3 = cv2.cvtColor(img3,cv2.COLOR_GRAY2BGR) #Make a color image with grayscale
print(img3.shape)
print(result.shape)
add = cv2.add(img3,result) #Combine images

#Image display
cv2.imshow('all',img)
cv2.imshow('image',add)
cv2.imshow('result',result)
cv2.waitKey(0) #Wait until some key is pressed
cv2.destroyAllWindows() #Destroy all windows

Execution result

Now you can see what's different.

I can only find 8! !! !! !!

Please tell me two more ...

It was just an auxiliary tool.

I found 9 spot the difference in December