I get an error when I put opencv in pyautoGUI

Overview

To do RPA in python When using pyautogui There are times when you want to perform tolerances, grayscale recognition, etc. At that time, it will be necessary to insert opencv.

However, I couldn't use it after all. I will post the error details and workarounds.

error contents

pos = pyautogui.locateCenterOnScreen(GetImage(imgname), region=inputregion) This code worked, but suddenly stopped working. pip opencv-python and pos = pyautogui.locateCenterOnScreen(GetImage(imgname), region=inputregion, grayscale=True) I tried to make it correspond to grayscale by rewriting.

output


============================= test session starts =============================
platform win32 -- Python 3.8.5, pytest-6.0.2, py-1.9.0, pluggy-0.13.1
================================== FAILURES ===================================
    pos = pyautogui.locateCenterOnScreen(GetImage(imgname), region=inputregion,grayscale=True)
C:\tools\miniconda3\envs\rpa2\lib\site-packages\pyautogui\__init__.py:175: in wrapper
    return wrappedFunction(*args, **kwargs)
C:\tools\miniconda3\envs\rpa2\lib\site-packages\pyautogui\__init__.py:207: in locateCenterOnScreen
    return pyscreeze.locateCenterOnScreen(*args, **kwargs)
C:\tools\miniconda3\envs\rpa2\lib\site-packages\pyscreeze\__init__.py:400: in locateCenterOnScreen
    coords = locateOnScreen(image, **kwargs)
C:\tools\miniconda3\envs\rpa2\lib\site-packages\pyscreeze\__init__.py:360: in locateOnScreen
    retVal = locate(image, screenshotIm, **kwargs)
C:\tools\miniconda3\envs\rpa2\lib\site-packages\pyscreeze\__init__.py:340: in locate
    points = tuple(locateAll(needleImage, haystackImage, **kwargs))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
needleImage = array([[31, 31, 31, ..., 31, 31, 31],
       [31, 31, 31, ..., 31, 31, 31],
       [31, 31, 31, ..., 31, 31, 31],
    ...31, 31, 31, ..., 31, 31, 31],
       [31, 31, 31, ..., 31, 31, 31],
       [31, 31, 31, ..., 31, 31, 31]], dtype=uint8)
haystackImage = array([[60, 60, 60, ..., 60, 60, 60],
       [60, 60, 60, ..., 60, 60, 60],
       [60, 60, 60, ..., 60, 60, 60],
    ...16, 16, 16, ..., 16, 16, 16],
       [16, 16, 16, ..., 16, 16, 16],
       [16, 16, 16, ..., 16, 16, 16]], dtype=uint8)
grayscale = True, limit = 1, region = (0, 945.0, 240.0, 135.0), step = 1
confidence = 0.999
    def _locateAll_opencv(needleImage, haystackImage, grayscale=None, limit=10000, region=None, step=1,
                          confidence=0.999):
        """
        TODO - rewrite this
            faster but more memory-intensive than pure python
            step 2 skips every other row and column = ~3x faster but prone to miss;
                to compensate, the algorithm automatically reduces the confidence
                threshold by 5% (which helps but will not avoid all misses).
            limitations:
              - OpenCV 3.x & python 3.x not tested
              - RGBA images are treated as RBG (ignores alpha channel)
        """
        if grayscale is None:
            grayscale = GRAYSCALE_DEFAULT
    
        confidence = float(confidence)
    
        needleImage = _load_cv2(needleImage, grayscale)
        needleHeight, needleWidth = needleImage.shape[:2]
        haystackImage = _load_cv2(haystackImage, grayscale)
    
        if region:
>           haystackImage = haystackImage[region[1]:region[1]+region[3],
                                          region[0]:region[0]+region[2]]
E           TypeError: slice indices must be integers or None or have an __index__ method
C:\tools\miniconda3\envs\rpa2\lib\site-packages\pyscreeze\__init__.py:202: TypeError
=========================== short test summary info ===========================
FAILED test_iqctrl.py::test_iqcreate - TypeError: slice indices must be integ...
============================== 1 failed in 2.03s ==============================

environment

A list of python modules you are using.

conda install python
pip install pyautogui
pip install rope yapf black autopep8 pylint pytest
pip install tqdm
pip install pyperclip
pip install pyodbc
pip install cx_Oracle

I added opencv-python here.

The OS is Windows 10.

Conclusion

The _locateAll_opencv function has not been tested in python3.x and is still developing. I get an error such as "The index that slices the array must be an integer". I couldn't get involved because it's in the pyautogui module.

Workaround

Currently, either put up with 2.X or give up opencv. If neither is acceptable, you may need to give up grayscale and ambiguous image search.

If you can rewrite the _locateAll_opencv function itself, it would be helpful if you could pull request to Github.

Related materials

Recommended Posts

I get an error when I put opencv in pyautoGUI
I got an error when I put opencv in python3 with Raspberry Pi [Remedy]
When I get an error with PyInstaller
When I get an error with Pylint in Atom on Windows
I got an error when saving with OpenCV
When I get a chromedriver error in Selenium
I get an error when trying meinheld + WebSocket + mongodb
When you get an error in python scraping (requests)
In the Chainer tutorial, I get an error when importing a package. (mock)
I get an error when I put a Python plugin in Visual Studio Code under the pyenv environment
I get an error with import pandas.
I get an error when trying to install maec 4.0.1.0 with pip
I get an error with all yum commands
[Python] I want to know the variables in the function when an error occurs!
When I name the file flask.py in Flask, I get Import Error: cannot import name'Flask'
I get a java.util.regex.PatternSyntaxException when splitting a string in PySpark
When OpenCV hconcat gives an error for some reason
I got an error when using Tensorboard with Pytorch
I get an error when installing scipy on t2.micro because I don't have enough memory
What to do if you get an error when importing matplotlib in Python (Mac)
I want to get an error message in Japanese with django Password Change Form
I get an error when I try to raise Python to 3 series using pyenv on Catalina
I got an error in vim and zsh in Python 3.7 series
I get [Error 2055] when trying to connect to MySQL on Heroku
What to do if you get an error when running "certbot renew" in CakePHP environment
I got an error when I tried to process luigi in parallel on windows, but the solution
I get a UnicodeDecodeError in mecab-python3
Resolved an error when putting pygame in python3 on raspberry pi
I get an ImportError on appengine_config.py
I get a KeyError in pyclustering.xmeans
I got an AttributeError when mocking the open method in python
Put python, numpy, opencv3 in ubuntu14
I got an error when I ran composer global require laravel / installer
I got an SSL Error when I installed Anaconda in a new environment, so I solved it (Windows10, Anaconda3-2019.10)
I get an OS Error: [Errno 8] Exec format error when running a Flask application with a python command
An error occurred in scikit-learn imputer
I put Arch in XPS 13 (7390) (Note)
I get a can't set attribute when using @property in python
I got an error when I ran meteor add accounts-password and got hooked
I got an error when trying to install Xgboost and its solution
What to do if you get an error when trying to load mnist
I got an unfamiliar error in Django: TypeError: resolve () got an unexpected keyword argument'strict'
When I try to upgrade pip, I get an infinite loop after failing to upgrade
What to do if you get an error when installing Dlib (Ubuntu)
[OSX] [pyenv] What to do when an SSL error occurs in pip
I put Python 2.7 in Sakura VPS 1GB.
The story of an error in PyOCR
I put Linux (Ubuntu) in VAIO SX14.
I can't get the element in Selenium!
What to do if you get an error when trying to send a message in tasks.loop () immediately after startup
I get an error ~ is zero, singular U when passing the covariance matrix from the Linear layer to MultivariateNormal
[Python] What to do if an error occurs in pip (pyinstaller, pyautogui, etc.)
Workaround if you get an error when trying to install PySide with pip
[python] What to do when an error occurs in send_keys of headless chrome
I get a strange window when I use the open directory dialog in Tkinter
What to do if you get an error when installing python with pyenv
[Python] What to check when you get a Unicode Decode Error in Django
If you get a long error when tabbing an interactive shell with Anaconda
Error that occurred in OpenCV3 and its solution Precautions when using OpenCV3 on Mac
I tried to get an image by scraping
When I try matplotlib in Python, it says'cairo.Context'