Test automation starting with L-Chika (5) Webcam and OCR integration

1.First of all

This is the 5th test automation series starting with L Chika.

Trial elemental technology of acceptance test automation with M5Stack and Python Webcam image capture, image cropping, OCR test runner that was done with interactive shell Incorporate into and automate.

Please see here for the articles so far.

  1. Test automation starting with L Chika
  2. Test automation starting with L-Chika (2) Improving maintainability of test scripts
  3. Test automation starting with L-Chika (3) Oscilloscope embedding
  4. Test automation starting with L-Chika (4) Improved maintainability of test scripts (2)

2. Test runner refurbishment

--Create the following commands and functions.

|command|argument|function |--------+--------+-------- |open_cam|Camera device number|Run OpenCV VideoCapture |close_cam|None|Release OpenCV Video Capture |capture_cam|file name|Capture camera image and save to file |crop_img|Input file name
Cutout position (vertical)
Cutout position (horizontal)
Output file name|Cut out the specified area from the image file and save it in the file |open_ocr|None|Make PyOCR available |exec_ocr|Image file name to be applied to OCR|OCR US ASCII strings from image files

--Redundant code has been removed along with the fix.

test-runner.py


#!/usr/bin/python3

#
# This software includes the work that is distributed in the Apache License 2.0
#

from time import sleep
import serial
import codecs
import csv
import sys
import visa
import cv2
from PIL import Image
import pyocr
import pyocr.builders

UNINITIALIZED = 0xdeadbeef

def open_cam(camera_number):
    return cv2.VideoCapture(camera_number)

def close_cam(cam):
    if cam != UNINITIALIZED:
        cam.release()

def capture_cam(cam, filename):
    if cam != UNINITIALIZED:
        _, img = cam.read()
        cv2.imwrite(filename, img)
        return True
    else:
        print("CAM Not Ready.")
        return False

def crop_img(filename_in, v, h, filename_out):
    img = cv2.imread(filename_in, cv2.IMREAD_COLOR)
    v0 = int(v.split(':')[0])
    v1 = int(v.split(':')[1])
    h0 = int(h.split(':')[0])
    h1 = int(h.split(':')[1])
    img2 = img[v0:v1, h0:h1]
    cv2.imwrite(filename_out, img2)
    return True

def open_ocr():
    ocr = pyocr.get_available_tools()
    if len(ocr) != 0:
        ocr = ocr[0]
    else:
        ocr = UNINITIALIZED
        print("OCR Not Ready.")
    return ocr

def exec_ocr(ocr, filename):
    try:
        txt = ocr.image_to_string(
            Image.open(filename),
            lang = "eng",
            builder = pyocr.builders.TextBuilder()
        )
    except:
        print("OCR Not Ready.")
    else:
        return txt

def serial_write(h, string):
    if h != UNINITIALIZED:
        string = string + '\n'
        string = str.encode(string)
        h.write(string)
        return True
    else:
        print("UART Not Initialized.")
        return False

def open_dso():
    rm = visa.ResourceManager()
    resources = rm.list_resources()
    #print(resources)
    for resource in resources:
        #print(resource)
        try:
            dso = rm.open_resource(resource)
        except:
            print(resource, "Not Found.")
        else:
            print(resource, "Detected.")
            return dso

    #Throw an error to caller if none succeed.
    return dso

def main():
    is_passed = True
    val = str(UNINITIALIZED)
    cam = UNINITIALIZED
    ocr = UNINITIALIZED
    dso = UNINITIALIZED
    uart = UNINITIALIZED

    with codecs.open('script.csv', 'r', 'utf-8') as file:
        script = csv.reader(file, delimiter=',', lineterminator='\r\n', quotechar='"')

        with codecs.open('result.csv', 'w', 'utf-8') as file:
            result = csv.writer(file, delimiter=',', lineterminator='\r\n', quotechar='"')

            for cmd in script:
                print(cmd)

                if "#" in cmd[0]:
                    pass

                elif cmd[0] == "sleep":
                    sleep(float(cmd[1]))

                elif cmd[0] == "open_cam":
                    cam = open_cam(int(cmd[1]))

                elif cmd[0] == "close_cam":
                    close_cam(cam)
                    cam = UNINITIALIZED

                elif cmd[0] == "capture_cam":
                    ret = capture_cam(cam, cmd[1])
                    if ret == False:
                        is_passed = False

                elif cmd[0] == "crop_img":
                    crop_img(cmd[1], cmd[2], cmd[3], cmd[4])

                elif cmd[0] == "open_ocr":
                    ocr = open_ocr()
                    if ocr == UNINITIALIZED:
                        is_passed = False

                elif cmd[0] == "exec_ocr":
                    try:
                        val = exec_ocr(ocr, cmd[1])
                    except:
                        is_passed = False
                    else:
                        cmd.append(str(val))

                elif cmd[0] == "open_dso":
                    try:
                        dso = open_dso()
                    except:
                        is_passed = False

                elif cmd[0] == "dso":
                    try:
                        if "?" in cmd[1]:
                            val = dso.query(cmd[1]).rstrip().replace(",", "-")
                            cmd.append(val)
                        else:
                            dso.write(cmd[1])
                    except:
                        is_passed = False

                elif cmd[0] == "open_uart":
                    try:
                        uart = serial.Serial(cmd[1], 115200, timeout=1.0, dsrdtr=1)
                    except:
                        is_passed = False

                elif cmd[0] == "send":
                    ret = serial_write(uart, cmd[1])
                    if ret == False:
                        is_passed = False

                elif cmd[0] == "rcvd":
                    try:
                        val = uart.readline().strip().decode('utf-8')
                        cmd.append(val)
                    except:
                        is_passed = False

                elif cmd[0] == "eval_str_eq":
                    if str(val) != str(cmd[1]):
                        is_passed = False

                elif cmd[0] == "eval_int_eq":
                    if int(val) != int(cmd[1]):
                        is_passed = False

                elif cmd[0] == "eval_int_gt":
                    if int(val) < int(cmd[1]):
                        is_passed = False

                elif cmd[0] == "eval_int_lt":
                    if int(val) > int(cmd[1]):
                        is_passed = False

                elif cmd[0] == "eval_dbl_eq":
                    if float(val) != float(cmd[1]):
                        is_passed = False

                elif cmd[0] == "eval_dbl_gt":
                    if float(val) < float(cmd[1]):
                        is_passed = False

                elif cmd[0] == "eval_dbl_lt":
                    if float(val) > float(cmd[1]):
                        is_passed = False

                else:
                    cmd.append("#")

                if is_passed == True:
                    cmd.append("OK")
                    print(cmd)
                    result.writerow(cmd)
                else:
                    cmd.append("NG")
                    print(cmd)
                    result.writerow(cmd)
                    close_cam(cam)
                    print("FAIL")
                    sys.exit(1)

    if is_passed == True:
        close_cam(cam)
        print("PASS")
        sys.exit(0)

main()

3. Test script

To check the crop_img command, see 3.2 Webcam Image Capture in Try Elemental Technologies for Acceptance Test Automation with M5Stack and Python. I am using the 123abc.bmp file generated by /pbjpkas/items/97b6e596a4009a71f1ab#32-webcam image capture).

script.csv


open_cam,1
capture_cam,abc1234.jpg
close_cam
crop_img,123abc.bmp,234:274,240:400,123abc_crop.bmp
open_ocr
exec_ocr,123abc_crop.bmp
eval_str_eq,123-abc

4. Test execution result

--The execution environment is Windows 10 (1909) + Anaconda3 2019.07 (Python 3.6.9 64-bit) [^ 1]. --You have saved the webcam image. --The image cut out by the crop_img command was subjected to OCR, and it was automatically confirmed that it matched the character string of the expected value.

result.csv


open_cam,1,OK
capture_cam,abc1234.jpg,OK
close_cam,OK
crop_img,123abc.bmp,234:274,240:400,123abc_crop.bmp,OK
open_ocr,OK
exec_ocr,123abc_crop.bmp,123-abc,OK
eval_str_eq,123-abc,OK

5. Conclusion

UART command transmission / reception, ON / OFF of embedded device switch, voltage measurement using Arduino A / D converter, oscilloscope built-in, Web camera built-in, image cropping, character string extraction and comparison using OCR It is now possible. I think that the possibility of system-level CI / CD for embedded devices and IoT devices has increased considerably.

[^ 1]: Install OpenCV, Tesseract-OCR, PyOCR, PIL and set environment variables of Tesseract-OCR Try elemental technology of acceptance test automation with M5Stack and Python Please see / items / 97b6e596a4009a71f1ab).

Recommended Posts

Test automation starting with L-Chika (5) Webcam and OCR integration
Test automation starting with L-Chika (3) Oscilloscope integration
[Automation] Manipulate mouse and keyboard with Python
GRPC starting with Go server and Dart client
System trading starting with Python 3: Investment and risk
Airtest, one question at a time. Unity app E2E test starting with Airtest and Poco