I made a stamp generator with GAN

Introduction

"Abolition of stamps" that Mr. Kono started with administrative reform On the other hand, the Hanko Parliamentary Union disagrees with "the most effective means of identity verification". (Reference ["Hanko Parliamentary" Imprint is effective for identity verification "Request for abolition" (https://www.nikkansports.com/general/news/202010070001057.html))

All right ... let's dare to ask! "** Does the stamp really have the ability to verify the identity? **" In order to verify this paradox, I made a "hanko generator" with GAN (Generative Adversarial Network).

• Click here for impatient people who want to see only the conclusion ⇒ Conclusion

Preparation

Sample data creation

--For the time being, I tried to make a handwritten signature and a stamp sample by force. --A4 paper is divided into 2 cm x 2 cm grids, and ** a total of 140 stamps of 14 x 10 are stamped ** ――Imprinted earnestly! Imprint! Imprint! Imprint! ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ (I feel like I understand the hardships of a very great person) --Scan → Erase grid lines with image processing software and trim margins

• did it! (Figure below)

――When you look at it like this, it's missing, tilted, or thin, and even if the same person pushes the same three-sentence format, there are quite a lot of variations.

--Next, divide each stamp ――It is difficult to do manual work with image processing software, and I want to match the data specifications with 64 * 64 pixels of MNIST, so I made a simple program. (Reference)

```python
import os
import glob
import numpy as np
import skimage.util
import skimage.io
import skimage.transform
out_size = 64 #Final output size (horizontal, vertical)
img = skimage.io.imread('data/low/hanko.jpg')
print(img.shape)
# (3307, 2340, 3)
#If the size is halfway and cannot be divided here, an error will occur, so resize
img_ = skimage.transform.resize(img, (64*14, 64*10), anti_aliasing=True)
img_gray = skimage.color.rgb2gray(img_) #Grayscale conversion
print(img_gray.shape)
#(3307, 2340)
blocks = skimage.util.view_as_blocks(img_gray, (img_gray.shape[0]//14, img_gray.shape[1]//10)).squeeze()
print(blocks.shape)
# (14, 10, 64, 64)
path_to_out = 'data/low/hanko/' #Output destination
os.makedirs(path_to_out, exist_ok=True) #Create if not
file_list = glob.glob(path_to_out + "hanko*jpg") #Existence check & delete
for f in file_list: os.remove(f)
idx = 0
for i in range(14):
    for j in range(10):
        skimage.io.imsave(path_to_out + 'hanko{:0=3}.jpg'.format(idx), blocks[i,j])
        idx += 1
```

Implementation of GAN

--The program is basically the same as the one introduced in the following books. (Detailed principles of the contents are omitted here) -"Learn while making! Development deep learning by PyTorch" --Chapter 5 Image Generation by GAN (DCGAN, Self-Attention GAN) --Book: https://www.amazon.co.jp/dp/4839970254 --Sample program: https://github.com/YutaroOgawa/pytorch_advanced/tree/master/3_semantic_segmentation

--Partially changed as below --Rewrite the folder path appropriately and adjust the number of data samples. ――In the case of the original MNIST (handwritten digit data set), it was 200 sheets, but this time it is 140 sheets, which is 60 less. Is it fine···

  def make_datapath_list():
    train_img_list = list()
    img_num = 140 #The number of samples
    for img_idx in range(img_num):
        img_path = "../data/low/hanko/hanko{:0=3}.jpg ".format(img_idx)
        train_img_list.append(img_path)
    return train_img_list

--Assuming trial and error, the learning execution part is slightly modified so that the number of epochs can be obtained from the command argument.

  import sys
  args = sys.argv
  num_epochs = int(args[1]) if len(args)>1 else 300 
  G_update, D_update = train_model(
      G, D, dataloader=train_dataloader, num_epochs=num_epochs)

Learning & execution

--Now, you're ready! let's try it. --This time, I tried two types of programs, "** DC GAN ", which is a typical GAN, and " SAGAN **", which is an improved version of it. -(Reference) Execution environment

• CPU : Intel(R) Xeon(R) CPU E3-1225 v6 @ 3.30GHz
• OS : Windows 10 Pro for Workstations
• GPU : NVIDIA Quadro P4000

DCGAN (Deep Convolutional GAN) implementation

• Source code
  • https://github.com/ishihara-jp/HankoGenerator/tree/main/DCGAN
• Execution result

--The upper row is the real one (learning sample), the lower row is the fake (generated sample) ――It's not so good ... I think it was a disaster because there were too many variations and too few samples. ――Next, let's try SAGAN considering locality and globality.

SAGAN (Self-Attention GAN) implementation

• Source code

• https://github.com/ishihara-jp/HankoGenerator/tree/main/SAGAN

• Execution result ――Oh, I can see a little! (Learning time is about 10 minutes on GPU)

  

--Look at the Self-Attention Map at this time (the darker the color, the more you are paying attention to when generating the image).

――Somehow, it seems that it is igniting around the stamp. --Try increasing epoch (number of learning trials) from 300 to 500

-I can see it! You can see it! ... Then, it's double 1000 epoch!

--The contrast of Self-Attention Map has become clear.

――It seems that the dark part on the upper right of the stamp is working. Perhaps you are seeing the habit of landing from the upper right when you press the stamp (arbitrary interpretation)

Conclusion

Now it's a problem. A and B stamps. Which is the real one?

Can you spot it?

You may be able to tell by the color depth, but it may be indistinguishable at first glance.

Perhaps it is said that "If you duplicate with this kind of image processing, it will be one shot", The stamp generator I made this time just inputs a random number The miso is where you can make "** infinite fake stamps **".

It's not just a copy, but just like my push variation, GAN creates a variety of stamps.

This time, it is this level with a small number of samples prepared in the foreground miso and a basic program. The latest GAN, Deepfake, is for real and fake faces, even when targeting the "face", which is said to have the highest human discrimination. It has reached a level where it is almost impossible to distinguish.

So, I came to the conclusion that "** In the AI era, Hanko's proof of identity is already doubtful **".

(Don't make a naive tsukkomi that you can tell by ink or red meat!)

By the way, the correct answer is B. A was a fake generated by GAN.

Next time preview?

Even if the human eye can be deceived, it may be possible to detect it in computer calculations. Therefore, I would like to calculate the similarity between real and fake stamp images using "feature point matching" used in fingerprint judgment. I will publish an article if there is a need.