I wanted to convert my face photo into a Yuyushiki style.

1.First of all

This article is the 24th day article of Yuyushiki Advent Calender 2019.

2. What you want to do

Last year's Advent Calender tried automatic coloring of black and white cartoons with CycleGAN. (Try to color manga with CycleGAN-Yuyushiki as an example-) CycleGAN is good at style conversion such as changing colors and textures, but it is not good at shape conversion.

Therefore, this year, using U-GAT-IT, which is a GAN that realizes both style conversion and shape conversion, a Yuyushiki-style illustration from a face photo Let's generate.

The image looks like this ↓ example.png (Quoted from mantan-web)

  1. U-GAT-IT
model.png

U-GAT-IT can also perform shape conversion, which CycleGAN was not good at.

The following are the results published by the author in the paper. The leftmost photo is the original photo, but you can see that the changes that accompany the shape conversion of cats to dogs are also done well. result.png

All figures are taken from U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation

4. Yuyushiki

スクリーンショット 2019-12-23 23.08.48.png [Yuyushiki](https://www.yuyushiki.net) is a four-frame manga by Komata Mikami, who is serializing in Manga Time Kirara.

5. Learning

The learning was done at Google Colaboratory. To learn, from this repository, paste it into the notebook cell in the order of util-> ops-> UGATIT-> main.

However, when executing with Notebook, parser will cause an error, so change it as follows.

#Comment out the following
#parser = argparse.ArgumentParser(description=desc)
#Omitted
#parser.add_argument(...)

#Add the following
!pip install easydict
import easydict
args = easydict.EasyDict({
       'phase': 'train',
       'light': False,
       'dataset': 'yuyu',
       ###The following is omitted
})

Place the dataset directly under the current directory as shown below.

└─ dataset
    └─ yuyu
        ├─ trainA  #Photo of a woman for learning(Diverted from selfie2anime)
        ├─ trainB  #Yuyushiki character pictures for learning
        ├─ testA   #For testing(ry
        └─ testB   #For testing(ry

Now, when you're ready, start learning!

This model is very large, and even with a dataset of only about 100 sheets, the error that it does not fit in memory frequently occurred. .. In such a case, if you set'light' in args to True, it will learn with the light version. (It's a trade-off with accuracy, but ...) This time, I learned 1500 epoch of 100 iteration, which is a little overfitting.

** ○ Result ** Original photo / conversion result photo. female12.pngfemale12.png female77.pngfemale77.png female66.pngfemale66.png

It's a Yuyushiki-style painting, but it's subtle ...

6. At the end

This time, I didn't get the results I expected. I'm sorry I couldn't bring out the performance of the model well.

I think the main factor is that the training data set is too small. I'm feeling the limits of resources. .. (I want someone to make a Yuyushiki dataset)
Tomorrow is the last day. Have a nice Christmas, everyone!

Recommended Posts

I wanted to convert my face photo into a Yuyushiki style.
I tried machine learning to convert sentences into XX style
I want to INSERT a DataFrame into MSSQL
I made a program to convert images into ASCII art with Python and OpenCV
I made a code to convert illustration2vec to keras model
I wanted to modify Django's admin site a bit
[Markov chain] I tried to read a quote into Python.
I made my own OSS because I wanted to contribute to it
I made a script to say hello at my Koshien
[Twitter] I want to make the downloaded past tweets (of my account) into a beautiful CSV
I tried to convert a Python file to EXE (Recursion error supported)
numpy: I want to convert a single type ndarray to a structured array
I wanted to solve the ABC164 A ~ D problem with Python
I want to check the position of my face with OpenCV!
[Introduction to StyleGAN] I played with "A woman transforms into Mayuyu" ♬
[Django] I wanted to test when POSTing a large file [TDD]
Convert A4 PDF to A3 every 2 pages
I wanted to evolve cGAN to ACGAN
Convert a string to an image
A tool to convert Juniper config
Hash chain I wanted to avoid (1)
I wanted to generate a sentence like "Fucking Deca Rashomon" (past form)
I wrote a code to convert quaternions to z-y-x Euler angles in Python
I made a web application that converts photos into Van Gogh's style
I tried to make a face diagnosis AI for a female professional golfer ①
I tried to create a program to convert hexadecimal numbers to decimal numbers with python
Mayungo's Python Learning Episode 6: I tried to convert a character string to a number
I tried to make a face diagnosis AI for a female professional golfer ②
I came up with a way to make a 3D model from a photo.
I made a CLI tool to convert images in each directory to PDF
I want to convert a table converted to PDF in Python back to CSV
[To Twitter gentlemen] I wrote a script to convert .jpg-large to .jpg at once.
I made a network to convert black and white images to color images (pix2pix)
I made a script in python to convert .md files to Scrapbox format
I made a tool to convert Jupyter py to ipynb with VS Code