I analyzed the image of the Kyudo scoring book (a booklet that records the results of the hits). (Google Colaboratory)
What is Kyudo scoring book?
This is a booklet that records whether the arrow you shot was on target or missed.
There are various recording methods depending on the group, but this time I challenged the analysis of the vermilion ink marked with a circle.
It's a poor code, but I hope it helps someone.
Structure of this program
Use of google collaboratery
I created the program on the assumption that ** google collaboratery ** will be used.
https://colab.research.google.com/notebooks/welcome.ipynb?hl=ja
google collab is a Jupyter notebook environment that runs entirely in the cloud. You don't have a computer because you can use it for free without any settings! At that time, I didn't put python on my computer! Even in that case, it is easy to use.
In addition, it takes time to install the library on my personal computer, but since most of the libraries are already installed in google collab, it is a surprisingly big advantage that it can be executed by brain death.
This time I will read and write images, so I decided to link it with ** google Drive **.
Electronic data conversion plan
The design of this code is as follows.
** 1. Link with Google Drive, create folders 2. Get image, change image size 3. Recognize the frame of the scoring book 4. Recognize the red circle in the scoring book 5. Arrange the position information of the circle 6. Write to Excel **
Recognition of the outer frame of the booklet
A straight line was recognized under certain conditions, and the vertical and horizontal lines at the edge of the image among the recognized straight lines were specified as the return values.
--Uses straight line detection by Hough transform.
http://labs.eecs.tottori-u.ac.jp/sd/Member/oyamada/OpenCV/html/py_tutorials/py_imgproc/py_houghlines/py_houghlines.html
detect_line.py
def resize_im(self, im): #Fixed image size
# --------------------------------------------
size = self.x_pixel
h, w, c = im.shape
width,height = size, round(size * ( h / w ))
im_resize = cv2.resize(im,(width, height))
return im_resize
def detect_line(self): #Detect frames
# -----------------------------------------
im = cv2.imread(path_Now_Projects + self.FileName)
im_resize = self.resize_im(im)
# parameter
G = 1 + 2 * self.nomalization(10)
T1 = 1 + 2 * self.nomalization(7)
T2 = 1 + 2 * self.nomalization(2)
#Process the image (noise removal, blurring, binarization)
im_denoise = cv2.fastNlMeansDenoising(im_resize)
im_gray = cv2.cvtColor(im_denoise, cv2.COLOR_BGR2GRAY)
im_gau = cv2.GaussianBlur(im_gray,(G,G),0)
im_th = cv2.adaptiveThreshold(im_gau, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY,T1,T2)
if detail2 == True:
cv2.imwrite(self.path_project + self.ImName + "_th.jpg ", im_th)
#Extract a straight line.
imgEdge = cv2.Canny(im_th,50,150,apertureSize = 3) #Edge detection by Canny method
minLineLength = self.nomalization(200) #Threshold for the length of the straight line to be detected (depending on the number of pixels)
maxLineGap = self.nomalization(20) #The longest distance between straight lines that can be regarded as continuous (depending on the number of pixels)
th = self.nomalization(50)
lines = cv2.HoughLinesP(imgEdge,2,np.pi/180,th,minLineLength=minLineLength,maxLineGap=maxLineGap) #Detection of straight lines by Huff approximation
#While drawing the straight line in blue, select the straight line of the frame.
im_line = im_resize
frame_left,frame_under, frame_over, frame_right = [10000]*4,[1]*4, [10000]*4, [1]*4 #Initial value setting
#Draw all straight lines
for i in range(len(lines)):
for x1,y1,x2,y2 in lines[i]:
cv2.line(im_line,(x1,y1),(x2,y2),(255,0,0),2)
#Sorting straight lines of frames
if frame_left[0] > x1 and abs(y1-y2) >3*abs(x1-x2) : #Vertical line with the smallest x coordinate
frame_left = [x1,y1,x2,y2]
if frame_under[1] < y1 and 3*abs(y1-y2) < abs(x1-x2) : #Horizontal line with the largest y coordinate
frame_under = [x1,y1,x2,y2]
if frame_over[1] > y1 and 3*abs(y1-y2) < abs(x1-x2) : #Horizontal line with the smallest y coordinate
frame_over = [x1,y1,x2,y2]
if frame_right[0] < x1 and abs(y1-y2) >3*abs(x1-x2) : #Vertical line with the largest x coordinate
frame_right = [x1,y1,x2,y2]
#Draw a straight line indicating the frame in green.
cv2.line(im_line,(frame_left[0], frame_left[1]),(frame_left[2], frame_left[3]),(0,255,0),2)
cv2.line(im_line,(frame_under[0], frame_under[1]),(frame_under[2], frame_under[3]),(0,255,0),2)
cv2.line(im_line,(frame_over[0], frame_over[1]),(frame_over[2], frame_over[3]),(0,255,0),2)
cv2.line(im_line,(frame_right[0], frame_right[1]),(frame_right[2], frame_right[3]),(0,255,0),2)
if detail2 == True: #Save the image for debugging.
cv2.imwrite(self.path_project + self.ImName + "_line.jpg ", im_line)
return frame_left, frame_under, frame_over, frame_right
Find the intersection of each of the four straight lines
get_4point.py
def cross_point(self, p1, p2): #Derivation of the intersection of two straight lines passing through two points
# -----------------------------------------------------------
return solve( [ solve(p1,[1,1]), solve(p2,[1,1]) ], [1,1] )
def get_4point(self, f_under, f_left,f_over,f_right):#Get 4 intersections of 4 straight lines passing through 2 points
# ------------------------------------------------------------------------------------
f_under = np.array([f_under[0:2], f_under[2:4]])
f_left = np.array([f_left[0:2], f_left[2:4]])
f_over = np.array([f_over[0:2], f_over[2:4]])
f_right = np.array([f_right[0:2], f_right[2:4]])
UL = self.cross_point(f_under, f_left)
OL = self.cross_point(f_over , f_left)
UR = self.cross_point(f_under, f_right)
OR = self.cross_point(f_over, f_right)
return [OL, OR, UL, UR]
Rectangle trimming at any 4 points
transform_by4.py
def transform_by4(self, points):#Trim from any 4 points to a rectangle
# --------------------------------------------------------------
im = cv2.imread(path_Now_Projects + self.FileName)
im_resize = self.resize_im(im)
points = sorted(points, key=lambda x:x[1]) #Sort in ascending order of y.
top = sorted(points[:2], key=lambda x:x[0]) #The first two are on the square. You can also see the left and right by sorting by x.
bottom = sorted(points[2:], key=lambda x:x[0], reverse=True) #The latter two are below the rectangle. Also sorted by x.
points = np.array(top + bottom, dtype='float32') #Rejoin the two separated parts.
width = max(np.sqrt(((points[0][0]-points[2][0])**2)*2), np.sqrt(((points[1][0]-points[3][0])**2)*2))
height = max(np.sqrt(((points[0][1]-points[2][1])**2)*2), np.sqrt(((points[1][1]-points[3][1])**2)*2))
dst = np.array([
np.array([0, 0]),
np.array([width-1, 0]),
np.array([width-1, height-1]),
np.array([0, height-1]),
], np.float32)
trans = cv2.getPerspectiveTransform(points, dst) #If you pass the correspondence between the coordinates before conversion and the coordinates after conversion, a perspective transformation matrix will be created.
im_trimming = cv2.warpPerspective(im_resize, trans, (int(width), int(height))) #Cut out using the perspective transformation matrix.
if detail2 == True:
cv2.imwrite(self.path_project + self.ImName +'_trimming.jpg', im_trimming)
return im_trimming
Extraction of red circle
The method of extracting the red color is troublesome, such as mask processing in a certain range of the hsv color space.
--Sites referenced in mask processing
https://note.nkmk.me/python-opencv-numpy-alpha-blend-mask/
https://www.blog.umentu.work/python-opencv3%E3%81%A7%E3%83%9E%E3%82%B9%E3%82%AF%E5%87%A6%E7%90%86%E3%81%97%E3%81%A6%E3%81%BF%E3%82%8B%EF%BC%88%E3%81%8A%E3%81%BE%E3%81%91%E3%81%82%E3%82%8A%EF%BC%89/
def detect_red(self, im_trimming):#Extract only red
# ------------------------------------------------
im = im_trimming
im_resize = self.resize_im(im)
#red(H is 0~30,150~180 range is red)Prepare a mask
hsv = cv2.cvtColor(im_resize, cv2.COLOR_BGR2HSV)
lower1 = np.array([150, 30, 100]) # HSV
upper1 = np.array([179, 255, 255]) # HSV
img_mask1 = cv2.inRange(hsv, lower1, upper1)
lower2 = np.array([0, 30, 100]) # HSV
upper2 = np.array([30, 255, 255]) # HSV
img_mask2 = cv2.inRange(hsv, lower2, upper2)
#Combine two red masks
mask = cv2.bitwise_or(img_mask1, img_mask2)
#Put on a mask and leave only the red circle
im_red = cv2.bitwise_and(im_resize, im_resize, mask=mask)
if detail2 == True: #Save image for debugging
cv2.imwrite(self.path_project + self.ImName + "_red.jpg ", im_red)
return im_red
Circle detection
Circle detection is performed on the image extracted only in red. If you make a mistake in setting the conditions, every pattern will be recognized as a circle, so the conditions are essential.
This time, as with straight line detection, the Huff function is used.
--Circle detection by Hough transform
http://labs.eecs.tottori-u.ac.jp/sd/Member/oyamada/OpenCV/html/py_tutorials/py_imgproc/py_houghlines/py_houghlines.html
detect_circle.py
def detect_circle(self, im_trimming):#Get the position of the circle
# ---------------------------------------------------
# parameter
minD = self.nomalization(58)
p2= self.nomalization(12)
minR = self.nomalization(30)
maxR = self.nomalization(36)
Lx0 = self.nomalization(10)
Ly0 = self.nomalization(86)
Lx = self.nomalization(90)
Ly = self.nomalization(72)
#Detect a circle from the red-extracted image.
im_red = self.detect_red(im_trimming)
im_gray = cv2.cvtColor(im_red,cv2.COLOR_BGR2GRAY)
#Set the size of the circle to be detected before and after the size of the circle based on the number of pixels
circles = cv2.HoughCircles(im_gray,
cv2.HOUGH_GRADIENT,
dp=1,
minDist = minD, #Intervals per circle that allow detection
param1=1,
param2=p2, #Detection threshold
minRadius=minR, #Minimum radius to detect
maxRadius=maxR) #Maximum radius to detect
All chords
kaiseki.py
# coding: utf-8
#Digitize the photo of the hit in the scoring book
# ________________________________
#Output user settings"True"or"False"
detail1 = True
detail2 = True
# 1 =Image for confirmation
# 2 =Image for parameter adjustment
# ________________________________
#import list
import numpy as np
from numpy.linalg import solve
import os
import cv2
import sys
import pandas as pd
import openpyxl as excel
from pandas import ExcelWriter
import matplotlib.pyplot as plt
#Works with google drive
from google.colab import drive
drive.mount('/content/drive')
#path list
path_Now_Projects = 'drive/My Drive/OU_kyudo/Now_Projects/'
path_Past_Projects = 'drive/My Drive/OU_kyudo/Past_Projects/'
#Create a folder
def make_folder(path):
if os.path.exists(path)==False:
os.mkdir(path)
make_folder(path_Now_Projects)
make_folder(path_Past_Projects)
#Get the image name
files = []
for filename in os.listdir(path_Now_Projects):
if os.path.isfile(os.path.join(path_Now_Projects, filename)): #Get only files
files.append(filename)
if len(files)==0:
print("Image Now_Put it in the Projects folder.")
sys.exit()
#=============================
#<<<<<< C l a s s >>>>>>>>>>
class Tekichu(object): #initialize.
# --------------------------------
def __init__(self):
#Image name (with extension)
self.FileName = ""
#Image name (without extension)
self.ImName, self.ext = "",""
#project name and its path name
self.project = ""
self.path_project = ""
#Number of pixels in the horizontal direction of the image
self.x_pixel = 1800
def set_variable(self, file): #Set the name of the image
# ----------------------------------------------------
#project name and its path name
self.project = input("image("+ file +") Enter the project name: ")
self.path_project = "drive/My Drive/OU_kyudo/Now_Projects/" + self.project +"/"
#Create a folder with the project name
if os.path.exists(self.path_project)==False:
os.mkdir(self.path_project)
#Image name (with extension)
self.FileName = file
#Image name (without extension)
self.ImName, self.ext = os.path.splitext(file)
#Normalize parameters that fluctuate with pixels using reference values
def nomalization(self, val):
return int(self.x_pixel *(val / 1200))
def resize_im(self, im): #Fixed image size
# --------------------------------------------
size = self.x_pixel
h, w, c = im.shape
width,height = size, round(size * ( h / w ))
im_resize = cv2.resize(im,(width, height))
return im_resize
def detect_line(self): #Detect frames
# -----------------------------------------
im = cv2.imread(path_Now_Projects + self.FileName)
im_resize = self.resize_im(im)
# parameter
G = 1 + 2 * self.nomalization(10)
T1 = 1 + 2 * self.nomalization(7)
T2 = 1 + 2 * self.nomalization(2)
#Process the image (noise removal, blurring, binarization)
im_denoise = cv2.fastNlMeansDenoising(im_resize)
im_gray = cv2.cvtColor(im_denoise, cv2.COLOR_BGR2GRAY)
im_gau = cv2.GaussianBlur(im_gray,(G,G),0)
im_th = cv2.adaptiveThreshold(im_gau, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY,T1,T2)
if detail2 == True:
cv2.imwrite(self.path_project + self.ImName + "_th.jpg ", im_th)
#Extract a straight line.
imgEdge = cv2.Canny(im_th,50,150,apertureSize = 3) #Edge detection by Canny method
minLineLength = self.nomalization(200) #Threshold for the length of the straight line to be detected (depending on the number of pixels)
maxLineGap = self.nomalization(20) #The longest distance between straight lines that can be regarded as continuous (depending on the number of pixels)
th = self.nomalization(50)
lines = cv2.HoughLinesP(imgEdge,2,np.pi/180,th,minLineLength=minLineLength,maxLineGap=maxLineGap) #Detection of straight lines by Huff approximation
#While drawing the straight line in blue, select the straight line of the frame.
im_line = im_resize
frame_left,frame_under, frame_over, frame_right = [10000]*4,[1]*4, [10000]*4, [1]*4 #Initial value setting
#Draw all straight lines
for i in range(len(lines)):
for x1,y1,x2,y2 in lines[i]:
cv2.line(im_line,(x1,y1),(x2,y2),(255,0,0),2)
#Sorting straight lines of frames
if frame_left[0] > x1 and abs(y1-y2) >3*abs(x1-x2) : #Vertical line with the smallest x coordinate
frame_left = [x1,y1,x2,y2]
if frame_under[1] < y1 and 3*abs(y1-y2) < abs(x1-x2) : #Horizontal line with the largest y coordinate
frame_under = [x1,y1,x2,y2]
if frame_over[1] > y1 and 3*abs(y1-y2) < abs(x1-x2) : #Horizontal line with the smallest y coordinate
frame_over = [x1,y1,x2,y2]
if frame_right[0] < x1 and abs(y1-y2) >3*abs(x1-x2) : #Vertical line with the largest x coordinate
frame_right = [x1,y1,x2,y2]
#Draw a straight line indicating the frame in green.
cv2.line(im_line,(frame_left[0], frame_left[1]),(frame_left[2], frame_left[3]),(0,255,0),2)
cv2.line(im_line,(frame_under[0], frame_under[1]),(frame_under[2], frame_under[3]),(0,255,0),2)
cv2.line(im_line,(frame_over[0], frame_over[1]),(frame_over[2], frame_over[3]),(0,255,0),2)
cv2.line(im_line,(frame_right[0], frame_right[1]),(frame_right[2], frame_right[3]),(0,255,0),2)
if detail2 == True: #Save the image for debugging.
cv2.imwrite(self.path_project + self.ImName + "_line.jpg ", im_line)
return frame_left, frame_under, frame_over, frame_right
def cross_point(self, p1, p2): #Derivation of the intersection of two straight lines passing through two points
# -----------------------------------------------------------
return solve( [ solve(p1,[1,1]), solve(p2,[1,1]) ], [1,1] )
def get_4point(self, f_under, f_left,f_over,f_right):#Get 4 intersections of 4 straight lines passing through 2 points
# ------------------------------------------------------------------------------------
f_under = np.array([f_under[0:2], f_under[2:4]])
f_left = np.array([f_left[0:2], f_left[2:4]])
f_over = np.array([f_over[0:2], f_over[2:4]])
f_right = np.array([f_right[0:2], f_right[2:4]])
UL = self.cross_point(f_under, f_left)
OL = self.cross_point(f_over , f_left)
UR = self.cross_point(f_under, f_right)
OR = self.cross_point(f_over, f_right)
return [OL, OR, UL, UR]
def transform_by4(self, points):#Trim from any 4 points to a rectangle
# --------------------------------------------------------------
im = cv2.imread(path_Now_Projects + self.FileName)
im_resize = self.resize_im(im)
points = sorted(points, key=lambda x:x[1]) #Sort in ascending order of y.
top = sorted(points[:2], key=lambda x:x[0]) #The first two are on the square. You can also see the left and right by sorting by x.
bottom = sorted(points[2:], key=lambda x:x[0], reverse=True) #The latter two are below the rectangle. Also sorted by x.
points = np.array(top + bottom, dtype='float32') #Rejoin the two separated parts.
width = max(np.sqrt(((points[0][0]-points[2][0])**2)*2), np.sqrt(((points[1][0]-points[3][0])**2)*2))
height = max(np.sqrt(((points[0][1]-points[2][1])**2)*2), np.sqrt(((points[1][1]-points[3][1])**2)*2))
dst = np.array([
np.array([0, 0]),
np.array([width-1, 0]),
np.array([width-1, height-1]),
np.array([0, height-1]),
], np.float32)
trans = cv2.getPerspectiveTransform(points, dst) #If you pass the correspondence between the coordinates before conversion and the coordinates after conversion, a perspective transformation matrix will be created.
im_trimming = cv2.warpPerspective(im_resize, trans, (int(width), int(height))) #Cut out using the perspective transformation matrix.
if detail2 == True:
cv2.imwrite(self.path_project + self.ImName +'_trimming.jpg', im_trimming)
return im_trimming
def detect_red(self, im_trimming):#Extract only red
# ------------------------------------------------
im = im_trimming
im_resize = self.resize_im(im)
#red(H is 0~30,150~180 range is red)Prepare a mask
hsv = cv2.cvtColor(im_resize, cv2.COLOR_BGR2HSV)
lower1 = np.array([150, 30, 100]) # HSV
upper1 = np.array([179, 255, 255]) # HSV
img_mask1 = cv2.inRange(hsv, lower1, upper1)
lower2 = np.array([0, 30, 100]) # HSV
upper2 = np.array([30, 255, 255]) # HSV
img_mask2 = cv2.inRange(hsv, lower2, upper2)
#Combine two red masks
mask = cv2.bitwise_or(img_mask1, img_mask2)
#Put on a mask and leave only the red circle
im_red = cv2.bitwise_and(im_resize, im_resize, mask=mask)
if detail2 == True: #Save image for debugging
cv2.imwrite(self.path_project + self.ImName + "_red.jpg ", im_red)
return im_red
def detect_circle(self, im_trimming):#Get the position of the circle
# ---------------------------------------------------
# parameter
minD = self.nomalization(58)
p2= self.nomalization(12)
minR = self.nomalization(30)
maxR = self.nomalization(36)
Lx0 = self.nomalization(10)
Ly0 = self.nomalization(86)
Lx = self.nomalization(90)
Ly = self.nomalization(72)
#Detect a circle from the red-extracted image.
im_red = self.detect_red(im_trimming)
im_gray = cv2.cvtColor(im_red,cv2.COLOR_BGR2GRAY)
#Set the size of the circle to be detected before and after the size of the circle based on the number of pixels
circles = cv2.HoughCircles(im_gray,
cv2.HOUGH_GRADIENT,
dp=1,
minDist = minD, #Intervals per circle that allow detection
param1=1,
param2=p2, #Detection threshold
minRadius=minR, #Minimum radius to detect
maxRadius=maxR) #Maximum radius to detect
circle_position = [[0 for i in range(20)] for j in range(13)]
total_number = [0 for i in range(13)]
warning = False
if circles is not None:
circles = circles.squeeze(axis=0) #Get the center of the circle
im_circle = self.resize_im(im_trimming)
#Parameters according to the grid of the scoring book
x_level = [int(Lx0+i*Lx) for i in range(13)]
y_level = [int(Ly0+j*Ly) for j in range(21)]
#Draw all grids
for i in x_level:
cv2.line(im_circle,(i, 0),(i, int(self.x_pixel * 9/16)),(0,0,255),1)
for j in y_level:
cv2.line(im_circle,(0, j),(self.x_pixel, j),(0,0,255),1)
#Arrange the center position of the circle by comparing it with the grid
for cx, cy, r in circles:
#Draw the circumference and center of the circle.
cv2.circle(im_circle, (cx, cy), r, (0, 255, 0), 2)
cv2.circle(im_circle, (cx, cy), 2, (0, 255, 0), 2)
horizontal = int((cx-Lx0) // Lx)
vertical = int((cy-Ly0)// Ly)
#When the circle extends beyond the grid, anomalies are detected and responded
if vertical >= 20:
vertical = 19
warning = True
#Record in array
circle_position[horizontal][vertical] += 1
#Anomalies are recorded when two or more are detected in one grid.
if circle_position[horizontal][vertical] >= 2:
warning = True
if detail1 == True:
cv2.imwrite(self.path_project + self.ImName + "_circles.jpg ", im_circle)
#Calculate total hit
for i in range(13):
total_number[i] = np.sum(circle_position[i])
#Textification
for i in range(13):
for j in range (20):
if circle_position[i][j] == 1:
circle_position[i][j] = "○"
elif circle_position[i][j] == 0:
circle_position[i][j] = "・"
#Join
data = np.r_[np.array([total_number]), np.array(circle_position).T] #Combine so that the total is the 0th line and the hit is the 1st to 20th lines
df = pd.DataFrame(data)
#View results
if warning == True :
print("[Warning] There is an error in the result."+ self.FileName)
print(df)
return df
def tekichu_main(self):#Main program in class
# ------------------------------------------------
f_left, f_under , f_over, f_right = self.detect_line()
box_points = self.get_4point(f_left, f_under , f_over, f_right)
im_trimming = self.transform_by4(box_points)
df = self.detect_circle(im_trimming)
wb = excel.Workbook() #Create a new workbook
wb.save(self.path_project + self.project +".xlsx")
writer = ExcelWriter(self.path_project + self.project + '.xlsx')
df.to_excel(writer, sheet_name = self.ImName) #Write to excel
writer.save()
return df
#==================================
#>>>main program>>>>>>>>>>>>>>>
if __name__ == '__main__':
for i in range(len(files)):
tek1 = Tekichu()
tek1.set_variable(files[i])
df = tek1.tekichu_main()
print("Finished normally")
Execution procedure
I really wanted to analyze it with a picture taken properly
But it was impossible.
In order to analyze the position of the hit circle, it is necessary to understand the positional relationship of the grid in the scoring book.
I used a straight line detection function to recognize it, but ...
If there is a background, it will be detected in an unpredictable way. As expected, this can't be helped.
⇒ Have the image cropped so that the background does not enter
With this as a condition, we have made it possible to perform stable analysis.
Drive preparation
First, due to the specifications of this program, create a folder called "kyudo" and two folders called "New_Projects" and "Past_Projects" in it.
Since it is designed to process the image contained in "New_Projects", put the cropped image there.
Run button when ready! This turns around ...
I was instructed to mount the drive. The first time you open Google Collab
If you are new to how to mount the drive, I think you should read it.
If you know it, please skip it.
Now select your account.
This completes the mount. Then enter the project name.
If you execute it after entering the project name, the analysis result will be displayed and the data in the table will be saved in the drive.
It was better to let the user handle a part on the GUI screen
After that, I wrote the code to process with GUI (graphic user interface) using Tkinter without using Google Colab, but that was better in terms of accuracy and operation.
Well, this was a learning experience and I enjoyed it.
Recommended Posts