Make pyautogui [super-appropriate] compatible with multi-display environment Part1

Purpose

I want to use pyautogui even in a multi-display environment, or more accurately, I want to recognize images in a sub-display under a multi-display environment.

environment

python 3.8.5 pyautogui 0.9.50 pyscreez 0.1.26 OS windows only

investigation

Reference: Why pyautogui's locateOnScreen () does not support multiple displays In the above article, I found the following description.

PIL's imageGrab.grab () does not support dual display

Read this? I thought, I checked the definition of imageGrab.grab ().

def grab(bbox=None, include_layered_windows=False, all_screens=False, xdisplay=None):

all_screens=False

Check the arguments like that, I don't know for some time, but at least in python 3.8.5, the grab function seems to have changed.

Implementation details

When the OS is Windows, the following function calls image.grab () in locateOnScreen () of pyautogui.

def _screenshot_win32(imageFilename=None, region=None):
    """
    TODO
    """

    # TODO - Use the winapi to get a screenshot, and compare performance with ImageGrab.grab()
    # https://stackoverflow.com/a/3586280/1893164
    im = ImageGrab.grab()

    if region is not None:
        assert len(region) == 4, 'region argument must be a tuple of four ints'
        region = [int(x) for x in region]
        im = im.crop((region[0], region[1], region[2] + region[0], region[3] + region[1]))
    if imageFilename is not None:
        im.save(imageFilename)
    return im

So, I'll pass an argument to imageGrab.grab (), that's it.

def _screenshot_win32(imageFilename=None, region=None):
    """
    TODO
    """

    # TODO - Use the winapi to get a screenshot, and compare performance with ImageGrab.grab()
    # https://stackoverflow.com/a/3586280/1893164
    #im = ImageGrab.grab()
    im = ImageGrab.grab(all_screens=True)

    if region is not None:
        assert len(region) == 4, 'region argument must be a tuple of four ints'
        region = [int(x) for x in region]
        im = im.crop((region[0], region[1], region[2] + region[0], region[3] + region[1]))
    if imageFilename is not None:
        im.save(imageFilename)
    return im

result

Image recognition is now possible on the sub-display.

problem

The display is lined up

[1][2][3]

Or

[1][2]
[3][4]

It would be nice if they were lined up in a well-behaved manner like

 [2] [1] [3](Arrangement of my home)

When arranged like this, locateOnScreen returns the coordinates with LeftTop of [2] as (0,0), while the windows side requires the coordinates with LeftTop of [1] as (0,0). Will be.

This problem has an api that enumerates monitors and returns coordinates in win32api, so I would like to use it next time to solve it. Up to here for this time.