[Introduction to StyleGAN] I played with "The Life of a Man" ♬

To be clear, just by using Example, you can immediately reproduce the video that is often played (male ⇒ female conversion), and it is really amazing when you try it. This time, it is an introductory article that I tried only the first entrance. The reference is as follows. 【reference】 ① I tried using StyleGANTry mixing Yukichi Fukuzawa with StyleGanPuzer/stylegan-encoderNVlabs/stylegan Currently, styleGAN2 seems to be released, but I would like to take the first step of StyleGAN and get a feel for it. As a result, I was able to create the following video in almost one day, so I will write an article about what I was interested in. example17_18.gif

What i did

·environment ・ Run pretrained_example.py ・ Move the latent space ・ Gif animation

·environment

The Keras-gpu environment is broken because I installed Pytorch the other day, so I installed it from scratch. Machine; 1060 base OS;Windows10 The basis is as in the previous article. However, Chainer etc. will not be installed. When I finished building the environment, I got an error in Tensorflow. So, we have replaced it with Tensorflow = 1.13.1 as shown below. After replacing it, it doesn't work well, so I'm restarting it. 【reference】 ⑤ How to specify the build with conda install of Anaconda It worked stably with this. The evidence is as follows (the warning appears in the part of ..., but this time it has no effect, so it is deleted)

(keras-gpu) C:\Users\user\stylegan-encoder-master>python
Python 3.6.9 |Anaconda, Inc.| (default, Jul 30 2019, 14:00:49) [MSC v.1915 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
...
>>> print(tf.__version__)
1.13.1

・ Run pretrained_example.py

To make this work, Weights could not be downloaded in the code as in Reference ①. Therefore, I downloaded it from Reference ⑦ below and placed it in the same way as Reference ① in the Tensorflow version. Reference ⑥ seems to be interesting, but it is still difficult to understand, so I will postpone it next time. 【reference】 ⑥ Play StyleGAN !! ~ Image editing without additional learning ~pacifinapacific/StyleGAN_LatentEditor

So the first code is:

pretrained_example.py


# Copyright (c) 2019, NVIDIA CORPORATION. All rights reserved.
#
# This work is licensed under the Creative Commons Attribution-NonCommercial
# 4.0 International License. To view a copy of this license, visit
# http://creativecommons.org/licenses/by-nc/4.0/ or send a letter to
# Creative Commons, PO Box 1866, Mountain View, CA 94042, USA.

"""Minimal script for generating an image using pre-trained StyleGAN generator."""

import os
import pickle
import numpy as np
import PIL.Image
import dnnlib
import dnnlib.tflib as tflib
import config

def main():
    # Initialize TensorFlow.
    tflib.init_tf()
    # Load pre-trained network.
    #url = 'https://drive.google.com/uc?id=1MEGjdvVpUsu1jB4zrXZN7Y4kBBOzizDQ' # karras2019stylegan-ffhq-1024x1024.pkl
    #with dnnlib.util.open_url(url, cache_dir=config.cache_dir) as f:
    #    _G, _D, Gs = pickle.load(f)
    fpath = './weight_files/tensorflow/karras2019stylegan-ffhq-1024x1024.pkl'
    with open(fpath, mode='rb') as f:
        _G, _D, Gs = pickle.load(f)
        # _G = Instantaneous snapshot of the generator. Mainly useful for resuming a previous training run.
        # _D = Instantaneous snapshot of the discriminator. Mainly useful for resuming a previous training run.
        # Gs = Long-term average of the generator. Yields higher-quality results than the instantaneous snapshot.
    # Print network details.
    Gs.print_layers()
    # Pick latent vector.
    rnd = np.random.RandomState(5)
    latents = rnd.randn(1, Gs.input_shape[1])
    # Generate image.
    fmt = dict(func=tflib.convert_images_to_uint8, nchw_to_nhwc=True)
    images = Gs.run(latents, None, truncation_psi=0.7, randomize_noise=True, output_transform=fmt)
    # Save image.
    os.makedirs(config.result_dir, exist_ok=True)
    png_filename = os.path.join(config.result_dir, 'example.png')
    PIL.Image.fromarray(images[0], 'RGB').save(png_filename)

if __name__ == "__main__":
    main()

The essence of this code is to give a random number of the same size as the initial input value of Generator Gs below. latents = rnd.randn(1, Gs.input_shape[1]) It is the part that is converted to an image below using it. images = Gs.run(latents, None, truncation_psi=0.7, randomize_noise=True, output_transform=fmt) 【reference】 ⑧ Random number generation summary by Numpy The following output with this code. example.png

・ Move the latent space

The next thing to do is to change the random numbers in the above latents and output them. So, I tried it with the following code. (If only the main part is described)

def main():
    # Initialize TensorFlow.
    tflib.init_tf()
    fpath = './weight_files/tensorflow/karras2019stylegan-ffhq-1024x1024.pkl'
    with open(fpath, mode='rb') as f:
        _G, _D, Gs = pickle.load(f)
    # Print network details.
    Gs.print_layers()

    # Pick latent vector.
    rnd = np.random.RandomState(5) #5
    latents1 = rnd.randn(1, Gs.input_shape[1])
    latents2 = rnd.randn(1, Gs.input_shape[1])
    for i in range(1,101,4):
        latents = i/100*latents1+(1-i/100)*latents2
        # Generate image.
        fmt = dict(func=tflib.convert_images_to_uint8, nchw_to_nhwc=True)
        images = Gs.run(latents, None, truncation_psi=0.7, randomize_noise=True, output_transform=fmt)
        # Save image.
        os.makedirs(config.result_dir, exist_ok=True)
        png_filename = os.path.join(config.result_dir, 'example{}.png'.format(i))
        PIL.Image.fromarray(images[0], 'RGB').save(png_filename)

In other words, it is a random number, but continuous latents1 latents2 is calculated, and the parameters are changed between them to draw. With this operation, you can see how the female changes smoothly to male 1 and male 2. latents12.jpg Here, I thought about how to wander the latent space one step further. That is, replace that part with the following code. Now you can see what kind of image is hidden though it is part of the latent space.

    # Pick latent vector.
    rnd = np.random.RandomState(6) #5
    for i in range(1,101,4):
        latents = rnd.randn(1, Gs.input_shape[1])

I got the following: latents.jpg When this happens, you will want to select any person in this image and see the transition. So, I tried to draw a picture of a man of example73 and a child of example69, which seems to change the most. It happens to be next to each other ... The code is below.

    # Pick latent vector.
    rnd = np.random.RandomState(6) #5
    latents_=[]
    a=17
    b=18
    for i in range(1,101,4):
        latents = rnd.randn(1, Gs.input_shape[1])
        latents_.append(latents)
        
    for j in range(25):
        latents_mean=j/25*latents_[a]+(1-j/25)*latents_[b]
        # Generate image.
        fmt = dict(func=tflib.convert_images_to_uint8, nchw_to_nhwc=True)
        images = Gs.run(latents_mean, None, truncation_psi=0.7, randomize_noise=True, output_transform=fmt)
        # Save image.
        os.makedirs(config.result_dir, exist_ok=True)
        png_filename = os.path.join(config.result_dir, 'example{}.png'.format(j))
        PIL.Image.fromarray(images[0], 'RGB').save(png_filename)

The output result exceeded expectations as shown below. boy2chinren.jpg It's strange that there is a picture of a child really growing up as an uncle.

・ Gif animation

Finally, I want to make a gif animation like everyone is doing. To do this, put the following code at the end and the gif will be saved.

    s=50
    images = []
    for i in range(25):
        im = Image.open(config.result_dir+'/example'+str(i)+'.png') 
        im =im.resize(size=(512, 512), resample=Image.NEAREST)
        images.append(im)
    for i in range(24,0,-1):
        im = Image.open(config.result_dir+'/example'+str(i)+'.png') 
        im =im.resize(size=(512, 512), resample=Image.NEAREST)
        images.append(im)    
    
    images[0].save(config.result_dir+'/example{}_{}.gif'.format(a,b), save_all=True, append_images=images[1:s], duration=100*2, loop=0)    

The following result is changed to size = (256, 256) due to the publication size. Also, the output is s = 25, excluding the binary interpolation of 6.

random numberseed Bivalent interpolation Subspace
6 example17_18_256.gif example0_1_256.gif
5 example0_1_256.gif example0_1_256.gif

Summary

・ I think it can be moved if the environment can be constructed. ・ Great image interpolation

・ There are many possible applications for this app alone, so I'll try using my own images. ・ I would like to take on the challenge of so-called video in a more developmental way.

Recommended Posts

[Introduction to StyleGAN] I played with "The Life of a Man" ♬
[Introduction to StyleGAN] I played with "A woman transforms into Mayuyu" ♬
[Introduction to Pytorch] I played with sinGAN ♬
[Introduction to sinGAN-Tensorflow] I played with the super-resolution "Challenge Big Imayuyu" ♬
[Introduction to Python] How to sort the contents of a list efficiently with list sort
[Introduction to StyleGAN] I played with style_mixing "Woman who takes off glasses" ♬
I failed to install django with pip, so a reminder of the solution
I want to set a life cycle in the task definition of ECS
I tried to create a model with the sample of Amazon SageMaker Autopilot
[Introduction to Python] How to get the index of data with a for statement
I tried to make something like a chatbot with the Seq2Seq model of TensorFlow
[Introduction to system trading] I drew a Stochastic Oscillator with python and played with it ♬
I tried to find the entropy of the image with python
I tried to find the average of the sequence with TensorFlow
I made a function to check the model of DCGAN
[Python] I want to make a 3D scatter plot of the epicenter with Cartopy + Matplotlib!
I wrote a doctest in "I tried to simulate the probability of a bingo game with Python"
Save the result of the life game as a gif with python
I tried to automate the watering of the planter with Raspberry Pi
[Introduction to Python] How to split a character string with the split function
I want to output the beginning of the next month with Python
I tried to create a list of prime numbers with python
How to output the output result of the Linux man command to a file
[Introduction to Matplotlib] Axes 3D animation: I played with 3D Lissajous figures ♬
I wanted to solve the ABC164 A ~ D problem with Python
[Introduction] I want to make a Mastodon Bot with Python! 【Beginners】
[Introduction to StyleGAN] Unique learning of anime with your own machine ♬
I tried to expand the size of the logical volume with LVM
I want to check the position of my face with OpenCV!
From the introduction of JUMAN ++ to morphological analysis of Japanese with Python
[Introduction to RasPi4] I played with "Hiroko / Hiromi's poisonous tongue conversation" ♪
I tried to improve the efficiency of daily work with Python
I tried to make a mechanism of exclusive control with Go
I tried to unlock the entrance 2 lock sesame with a single push of the AWS IoT button
[Scikit-learn] I played with the ROC curve
I tried to predict the number of domestically infected people of the new corona with a mathematical model
I made a life game with Numpy
[Introduction to AWS] I played with male and female voices with Polly and Transcribe ♪
[Introduction to Udemy Python3 + Application] 47. Process the dictionary with a for statement
Various methods to numerically create the inverse function of a certain function Introduction
[Django] I made a field to enter the date with 4 digit numbers
I tried to get the authentication code of Qiita API with Python.
I want to sort a list in the order of other lists
I want to express my feelings with the lyrics of Mr. Children
I tried to automatically extract the movements of PES players with software
I tried to analyze the negativeness of Nono Morikubo. [Compare with Posipa]
[Introduction to Python] How to write a character string with the format function
I made a GAN with Keras, so I made a video of the learning process.
I tried to streamline the standard role of new employees with Python
I tried to visualize the text of the novel "Weathering with You" with WordCloud
I want to stop the automatic deletion of the tmp area with RHEL7
I tried to get the movie information of TMDb API with Python
I made a program to check the size of a file in Python
I made a mistake in fetching the hierarchy with MultiIndex of pandas
I tried to display the altitude value of DTM in a graph
[Introduction to Python] What is the method of repeating with the continue statement?
I tried to verify the result of A / B test by chi-square test
Python: I want to measure the processing time of a function neatly
I tried to predict the behavior of the new coronavirus with the SEIR model.
I made a function to see the movement of a two-dimensional array (Python)
I tried to make a thumbnail image of the best avoidance flag-chan! With RGB values ​​[Histogram] [Visualization]