Rotate sprites with OpenCV # 2 ~ Mastering cv2.warpAffine () ~

Introduction

The previous article "Rotate sprites with OpenCV" couldn't be completed during the summer vacation, so I managed to break it up using the weekend. However, the day after it was uploaded, a new idea came to me. After all the environment with many temptations is not good. You have to put yourself in a harsher environment to study. Workplace. </ strike>

Details of cv2.warpAffine ()

There was an interesting non-essential argument of cv2.warpAffine () that I omitted last time. Reference article: Draw another image on the image with OpenCV

--src Original image. Required. --M 2 * 3 transformation matrix. Required. --dsize Specify the output image size with a tuple of (width, height). Required. --dst background (outside the original image) image. The size should be the same as dsize. --flags Image complement method. The default value is cv2.INTER_LINEAR. Others such as cv2.INTER_NEAREST. --BorderMode Background (outside the original image) processing method. The default value is cv2.BORDER_CONSTANT. --BorderValue The color when borderMode = cv2.BORDER_CONSTANT. The default value is 0. See below.

Small-process the original image in advance

To make the result of image processing easy to understand, create a small function. Please note that only the minimum part is described.

python


def makeSampleImg(img4):   #Bring an RGBA 4-channel image
    h, w = img.shape[:2]
    #A as well as coloring the surroundings=The miso is to make it 255 (opaque)
    cv2.rectangle(img, (0,0), (w-1,h-1), (0,0,255,255), 1)
    return img

img_origin = cv2.imread(filename, -1) #RGBA image
img4 = makeSampleImg(img_origin)
img3 = img4[:, :, :3]  #Eliminate the A element and make an RGB image
The original image Small processing 3 channels
uchuhikoushi.png uchuhikoushi4w.png uchuhikoushi3w.png

Fill the background with a single color

Make the background a single color with borderMode = cv2.BORDER_CONSTANT. Since this is the default value of borderMode, there is no need to describe it. I'll try some.

Specify background color


M = cv2.getRotationMatrix2D((w/2,h/2), 30, 1)
img_rot31 = cv2.warpAffine(img3, M, (w, h), borderValue=(255,255,0))     #Foreground RGB, background RGB
img_rot32 = cv2.warpAffine(img3, M, (w, h), borderValue=(255,255,0,0))   #Foreground RGB, background RGBA (A component = 0)
img_rot33 = cv2.warpAffine(img3, M, (w, h))                              #Foreground RGB, no background specified
img_rot41 = cv2.warpAffine(img4, M, (w, h), borderValue=(255,255,0))     #Foreground RGBA, background RGB
img_rot42 = cv2.warpAffine(img4, M, (w, h), borderValue=(255,255,0,0))   #Foreground RGBA, background RGBA (A component = 0)
img_rot43 = cv2.warpAffine(img4, M, (w, h))                              #Foreground RGBA, no background specified
img_rot44 = cv2.warpAffine(img4, M, (w, h), borderValue=(255,255,0,255)) #Foreground RGBA, background RGBA (with A component)

** If the foreground is an RGB image **, even if you specify 4 channels of RGBA as the background color, the A component is ignored (img_rot32). It should be understood that if borderValue is not specified, the background will be black (img_rot33, or rather the default operation), but the background color will be (0,0,0).

img_rot31 img_rot32 img_rot33
uchuhikoushi31.png uchuhikoushi32.png uchuhikoushi33.png

** If the foreground is an RGBA image **, the output result will be 4 RGBA channels even if 3 RGB channels are specified as the background color. The A component given at this time is 0. Since A is opacity, not transparency, this is 0, which means that even if a BGR value that was not RGB ... is defined, it will be transparent as a result (img_rot41). Even if borderValue is not specified, the background becomes transparent (img_rot43), but it is easy to understand if you think that the background color will be (0,0,0,0). Of course, if you set a value other than 0 to the A component, it will be colored properly (img_rot44).

img_rot41 img_rot42 img_rot43 img_rot44
uchuhikoushi41.png uchuhikoushi42.png uchuhikoushi43.png uchuhikoushi44.png

Set an image as the background

If you set borderMode = cv2.BORDER_TRANSPARENT, you can specify the background image with dst. As is often the case with OpenCV, cv2.warpAffine () will process the background image specified by dst. If you need to keep the original image, you need to set dst = back.copy ().

Since it is TRANSPARENT, it becomes transparent, so I expected that it would be a transparent background if I did not specify dst, but it was not so easy. Oh! Somehow I want to eat salmon roe.

Specify background image


back = cv2.imread(back_name)  #RGB image of the same size as the foreground image
back4 = cv2.cvtColor(back, cv2.COLOR_BGR2BGRA) #Make an RGBA image A component is 255 instead of 0

M = cv2.getRotationMatrix2D((w/2,h/2), 30, 1)
img_rot35 = cv2.warpAffine(img3, M, (w, h), borderMode=cv2.BORDER_TRANSPARENT, dst=back.copy())   #Foreground RGB, background RGB
img_rot36 = cv2.warpAffine(img3, M, (w, h), borderMode=cv2.BORDER_TRANSPARENT, dst=back4.copy())  #Foreground RGB, background RGBA
img_rot37 = cv2.warpAffine(img3, M, (w, h), borderMode=cv2.BORDER_TRANSPARENT)                    #Foreground RGB, no background specified
img_rot45 = cv2.warpAffine(img4, M, (w, h), borderMode=cv2.BORDER_TRANSPARENT, dst=back.copy())   #Foreground RGBA, background RGB
img_rot46 = cv2.warpAffine(img4, M, (w, h), borderMode=cv2.BORDER_TRANSPARENT, dst=back4.copy())  #Foreground RGBA, background RGBA
img_rot47 = cv2.warpAffine(img4, M, (w, h), borderMode=cv2.BORDER_TRANSPARENT)                    #Foreground RGBA, no background specified

Both the foreground and background behaved as expected with the RGB image (img_rot35).

If I didn't specify a background image in the RGB foreground, the result changed every time I ran it (img_rot37). numpy has a function callednumpy.empty ()that creates an uninitialized array. The background image (or rather the numpy array) is probably created with the same specifications here as well. I'm not sure why img_rot36, which specified the RGBA background in the RGB foreground, turned into a black background with dust.

img_rot35 img_rot36 img_rot37 part 1 img_rot37 part 2
uchuhikoushi35.png uchuhikoushi36.png uchuhikoushi37.png uchuhikoushi37_2.png

The RGBA image (img_rot46) for both the foreground and background is as expected, but it is also a disappointing result. I was happy if the background was displayed in the transparent part of the foreground, but ** affine transformation is not that kind of thing **, so it can't be helped. The background of img_rot45, which specified an RGB background for the RGBA foreground, and img_rot47, which did not specify a background image for the RGBA foreground, became transparent. In both cases, it seems that 0 was added to the A element.

In short, the mundane conclusion is that it is wise not to use it unexpectedly.

img_rot45 img_rot46 img_rot47
uchuhikoushi45.png uchuhikoushi46.png uchuhikoushi47.png

Use for sprites

As mentioned above, referring to the article of my predecessor, I thought about pasting an RGBA image including transparency on the image, but it was impossible to conclude. However, the specifications that do not cause an error even if it extends beyond the background image are attractive. Therefore, I decided to combine the mask method that I have done so far with the affine transformation. Then ... it's done, super easy. No circumscribed quadrangle, no ROI, no hassle.

Source


import cv2

def makeSampleImg(img4):
    h, w = img4.shape[:2]
    cv2.rectangle(img4, (0,0), (w-1,h-1), (0,0,255,255), 1)
    return img4

def putSprite_Affine(back, front4, pos, angle=0, center=(0,0)):
    x, y = pos
    front3 = front4[:, :, :3]
    mask1 =  front4[:, :, 3]
    mask3 = 255- cv2.merge((mask1, mask1, mask1))
    bh, bw = back.shape[:2]

    M = cv2.getRotationMatrix2D(center, angle, 1)
    M[0][2] += x
    M[1][2] += y
    front_rot = cv2.warpAffine(front3, M, (bw,bh))
    mask_rot = cv2.warpAffine(mask3, M, (bw,bh), borderValue=(255,255,255))
    tmp = cv2.bitwise_and(back, mask_rot)
    result = cv2.bitwise_or(tmp, front_rot)
    return result

if __name__ == "__main__":
    filename_front = "uchuhikoushi.png "
    filename_back = "space.jpg "
    img_front = cv2.imread(filename_front, -1)
    img_front = makeSampleImg(img_front)  #Add a frame to the RGBA image (not required)
    img_back = cv2.imread(filename_back)

    pos = [(0, 50), (300,200), (400,400), (500,-50), (-100,1000)] #Upper left coordinate to put the image
    xc, yc = 140, 60  #Center of rotation of the foreground image
    angle = 0

    while True:
        img = img_back.copy()
        for x,y in pos:
            img = putSprite_Affine(img, img_front, (x,y), angle, (xc,yc))

            #Make sure it is portrayed correctly (not required)
            cv2.circle(img, (x,y), 5, (0,255,0), -1)       #Mark in the upper left of the foreground image
            cv2.circle(img, (x+xc,y+yc), 5, (0,0,255), -1) #Mark at the center of rotation
            cv2.putText(img, f"angle={angle}", (10,440), cv2.FONT_HERSHEY_SIMPLEX, 1, (0,0,255), 2)

        cv2.imshow("putSprite_Affine", img)

        key = cv2.waitKey(1) & 0xFF
        if key == ord("q"):
            break
        angle = (angle + 30) % 360

    cv2.destroyAllWindows()

Speaking of drawbacks, the mask image and RGB image are affine-transformed for the entire background image, so the amount of calculation is uselessly large ("[Overhead](https://ja.wikipedia.org/wiki/%E3%82%]" AA% E3% 83% BC% E3% 83% 90% E3% 83% BC% E3% 83% 98% E3% 83% 83% E3% 83% 89) seems to be large. "

Front in the middle of drawing_rot
front_rot.png
Mask in the middle of drawing_rot
mask_rot.png
result
Here, the previous anime GIF is posted, but in reality, the character with the red frame rotates.uchuhikoushi_anim_3.gif

Now let's see how this affects execution speed.

At the end

It's not over yet.

Recommended Posts

Rotate sprites with OpenCV # 2 ~ Mastering cv2.warpAffine () ~
Rotate sprites with OpenCV
Rotate sprites with OpenCV # 3 ~ Calculate by yourself without leaving it to others ~
Detect stoop with OpenCV
Binarization with OpenCV / Python
Data Augmentation with openCV
Easy TopView with OpenCV
Stumble with homebrew opencv3
Face recognition with Python's OpenCV
"Apple processing" with OpenCV3 + Python3
Try edge detection with OpenCV
Image editing with python OpenCV
Camera capture with Python + OpenCV
[Python] Using OpenCV with Python (Basic)
Binarize photo data with OpenCV
Loop video loading with opencv
Real-time edge detection with OpenCV
Face detection with Python + OpenCV
Get image features with OpenCV
Face recognition / cutting with OpenCV
Try OpenCV with Google Colaboratory
Cascade classifier creation with opencv
Using OpenCV with Python @Mac
Image recognition with Keras + OpenCV
Anime face detection with OpenCV