[Python] Specify the range from the image by dragging the mouse

Thing you want to do

"Specify a range from an image" that you often see with trimming tools, etc. I want to make it possible by dragging the mouse while displaying the guideline. ↓ The image looks like this ↓ 画像領域取得デモ.gif

environment

Windows10 Home Python3.8.2（32bit） Visual Studio Code 1.45.1

What I studied (flow)

Display any image in the tkinter window
Draw a rectangle on the displayed image
Transform the rectangle by dragging the mouse
Get the coordinate information of the rectangle

1. Display any image in the tkinter window

In this sample, we plan to draw a rectangle (guideline) for the screenshot. Therefore, first take a screenshot using screenshot () of the pyautogui library. Since the prepared image cannot be used as it is in tkinter, ** use ImageTk.PthoImage to convert it to a format that can be displayed in tkinter **. img_tk = ImageTk.PhotoImage(img) (Note that this conversion will give an error like "Converting too early!" Unless it is after Tk ()) (ImageTk is a Pillow method, so you need to import it separately)

After that, create a Canvas widget with tkinter and place the image in it.

canvas1 = tkinter.Canvas(root, bg="black", width=img.width, height=img.height)
canvas1.create_image(0, 0, image=img_tk, anchor=tkinter.NW)

Since Canvas does not automatically transform to fit the image, you need to specify the widget size to be the same as the image.

The option ʻanchorof create_image specifies "where to base the image to be placed". The value is written astkinter.` in north, south, east and west, and the image is N: North ⇒ north = top. Other than N, write W: left, S: bottom, E: right, CENTER: center. In the case of the example, NW = upper left of the image of img_tk will be placed at x coordinate = 0 and y coordinate = 0 (specified by the first and second arguments).

2. Draw a rectangle on the displayed image

First, specify what happens when you start dragging on Canvas.

canvas widget.bind("<ButtonPress-1>", <Callback function>)

Guideline drawing processing is performed in the called callback function. (Rectangle means rectangle)

Canvas widget.create_rectangle(X coordinate of start point,Y coordinate of start point,X coordinate of the end point,Y coordinate of the end point, outline="Line color" ,tag="Tag name")

Since the rectangle drawn later will be transformed, it is necessary to specify the tag name with the option tag.

If the argument event is specified when declaring the callback function, ʻevent.x, event.y` Since you can get the mouse coordinates at the time of clicking, set this value to the start point coordinates (the end point is appropriate)

3. Transform the rectangle by dragging the mouse

Next, specify the operation when the mouse moves while dragging with Canvas.

Canvas widget.bind("<Button1-Motion>", <Callback function>)

Describe the process of transforming the guideline in the called callback function.

Canvas widget.coords("Tag name of the shape to be transformed",X coordinate of start point,Y coordinate of start point,X coordinate of the end point,Y coordinate of the end point)

You can change the size of the drawn figure with. (Coords seems to mean coordination) When changing the rectangle size by dragging, I paid attention to the following two irregularities.

① What to do if dragged to the left or above the start point

If you set the current graphic coordinates to the coordinates of the start point without thinking about anything, It causes a bug when the mouse being dragged moves to the left or above the start point. .. (The coordinate information of the start point is rewritten to the coordinates after dragging) 画像領域取得失敗2デモ.gif Therefore, by storing the coordinate information in the grobal variable when the mouse is clicked and using that value as the start point coordinate at the time of redrawing, Prevented replacement ↓

#Event when dragging starts- - - - - - - - - - - - - - - - - - - - - - - - - - 
def start_point_get(event):
    global start_x, start_y #Declared to write to a global variable
　　　　：
(Drawing)
　　　　:
    #Store coordinates in global variables
    start_x, start_y = event.x, event.y

#Events being dragged- - - - - - - - - - - - - - - - - - - - - - - - - - 
def rect_drawing(event):
   　　 ：
    # "rect1"Redraw tag image
    canvas1.coords("rect1", start_x, start_y, event.x, event.y)

(2) What to do when the mouse goes out of the screen area

In this case, if the mouse goes out of the drawing area, the coordinates of the screen edge will be retained. Therefore, depending on whether the mouse coordinates are "drawing area (0 <x ≤ image width)", "outside the drawing area (x <0)", or "outside the drawing area (width <x)". You need to rewrite the end point coordinates. You can overlap two if statements, but this time I tried to shorten the code by using the min function. (If you rearrange it, you can use the max function)

    if event.x < 0:
        end_x = 0 #Set 0 if the acquired coordinates are 0 or less
    else:
        end_x = min(img.width, event.x) #Set the width of the image or the acquired coordinates, whichever is smaller

4. Get the coordinate information of the rectangle

Specify the operation so that the coordinates can be acquired when the drag is completed.

Canvas widget.bind("<ButtonRelease-1>", <Callback function>)

You can use the coords method described above to get the coordinates of the shape.

start_x, start_y, end_x, end_y =Canvas widget.coords("Tag name")

In this case it is difficult to display a full screen screenshot in the tk window, so I'm displaying a somewhat reduced version and drawing a rectangle on top of it. Therefore, if you want to get real size coordinates, you need to multiply the coordinates obtained by coords by the reduction magnification. This time, I used list comprehension notation and wrote the code as short as possible.

    start_x, start_y, end_x, end_y = [
        round(n * RESIZE_RETIO) for n in canvas1.coords("rect1")]

The above is a series of steps. Thank you for your hard work.

Completed code

import tkinter
import time
import pyautogui  #External library
from PIL import Image, ImageTk  #External library

RESIZE_RETIO = 2 #Regulation of reduction ratio

#Event when dragging starts- - - - - - - - - - - - - - - - - - - - - - - - - - 
def start_point_get(event):
    global start_x, start_y #Declared to write to a global variable

    canvas1.delete("rect1")  #Already"rect1"Delete any tag shape

    #Draw a rectangle on canvas1 (rectangle means rectangle)
    canvas1.create_rectangle(event.x,
                             event.y,
                             event.x + 1,
                             event.y + 1,
                             outline="red",
                             tag="rect1")
    #Store coordinates in global variables
    start_x, start_y = event.x, event.y

#Events being dragged- - - - - - - - - - - - - - - - - - - - - - - - - - 
def rect_drawing(event):

    #Processing when the mouse pointer while dragging goes out of the area
    if event.x < 0:
        end_x = 0
    else:
        end_x = min(img_resized.width, event.x)
    if event.y < 0:
        end_y = 0
    else:
        end_y = min(img_resized.height, event.y)

    # "rect1"Redraw tag image
    canvas1.coords("rect1", start_x, start_y, end_x, end_y)

#Event when the drag is released- - - - - - - - - - - - - - - - - - - - - - - - - - 
def release_action(event):

    # "rect1"Get the coordinates of the tag image back to the original scale
    start_x, start_y, end_x, end_y = [
        round(n * RESIZE_RETIO) for n in canvas1.coords("rect1")
    ]

    #Display the acquired coordinates
    pyautogui.alert("start_x : " + str(start_x) + "\n" + "start_y : " +
                    str(start_y) + "\n" + "end_x : " + str(end_x) + "\n" +
                    "end_y : " + str(end_y))

#Main processing- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
if __name__ == "__main__":

    #Acquisition of image to be displayed (screenshot)
    img = pyautogui.screenshot()
    #Image resizing because the screenshot image cannot be displayed
    img_resized = img.resize(size=(int(img.width / RESIZE_RETIO),
                                   int(img.height / RESIZE_RETIO)),
                             resample=Image.BILINEAR)

    root = tkinter.Tk()
    root.attributes("-topmost", True) #Always bring the tkinter window to the foreground

    #Image conversion so that it can be displayed with tkinter
    img_tk = ImageTk.PhotoImage(img_resized)

    #Drawing Canvas widget
    canvas1 = tkinter.Canvas(root,
                             bg="black",
                             width=img_resized.width,
                             height=img_resized.height)
    #Draw the acquired image on the Canvas widget
    canvas1.create_image(0, 0, image=img_tk, anchor=tkinter.NW)

    #Place Canvas widget and set various events
    canvas1.pack()
    canvas1.bind("<ButtonPress-1>", start_point_get)
    canvas1.bind("<Button1-Motion>", rect_drawing)
    canvas1.bind("<ButtonRelease-1>", release_action)

    root.mainloop()

Future plans

Now that we have an interface for specifying a range from an image, I would like to connect it to various tools. For example, a tool that "cuts out continuously from the screenshot screen ⇒ saves as a name at the current time", Tools such as "OCR from specified range". Not only screenshot screens, but also tools that process images on the clipboard may be interesting.

The site that I used as a reference

Transform the rectangle displayed on the Python canvas