This model maps the ** input image (higher dimension) ** to ** latent space (lower dimension) **, and this time based on this ** latent variable (lower dimension) **, the ** image ( The purpose is to generate high-dimensional) ** and minimize the error between this and the input image. After training, this model can generate a high-dimensional image again from a high-dimensional image through a low-dimensional space, so that ** a latent space that more characterizes the training image ** can be obtained. In other words, it has the role of dimensional compression and can be called a so-called non-linear principal component analysis. This is very convenient because it has various uses such as ** curse of dimensionality **, ** transfer learning **, and ** anomaly detection **.

Now, I will explain the structure of the model to be used.

• Image size is 1/2 times with one convolution and 2 times with deconvolution
• Stabilization of learning and acceleration of convergence
• The possible range of latent variables is (-∞, + ∞)
• The possible range of pixel values is [-1, +1]

Expecting these effects, we built the following network.

3. Data set

• CelebA (Large-scale CelebFaces Attributes) dataset
  http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html

This time, we will use the CelebA dataset, which is a dataset of 202,599 celebrity face images (color). The image size is 178 x 218 [pixel], which causes some inconvenience when deconvolution occurs, so this time I resized it to ** 64 x 64 [pixel] **. Of these, ** 90% (182,340) were used for learning images **, and ** 10% (20,259) were used for test images **.

When this is input to the previous model, the latent space becomes (C, H, W) = (512,1,1). If you input an image of 128 x 128 [pixel] or more, the intermediate layer becomes a spatial latent space.

Comparison

This time, I will mainly investigate ** "How much speed is different between C ++ and Python" **, but I would like to compare the speed and performance under the following 5 types of environments. ..

--CPU main operation - Python - C++ --GPU main operation - Python --Non-deterministic --Deterministic - C++

1. Difference in main unit (CPU or GPU)

• CPU
  Good at handling "serial" and "complex" instructions
• GPU
  Good at processing "parallel" and "simple" instructions

As you can see from the above features, in Deep Learning that handles images, GPU is overwhelmingly advantageous in terms of calculation speed.

(1) Implementation by Python

• When using CPU

CPU.py

device = torch.device('cpu')  #Use CPU

model.to(device)  #Move model to CPU
image = image.to(device)  #Move data to CPU

• When using GPU

GPU.py

device = torch.device('cuda')    #Use default GPU
device = torch.device('cuda:0')  #Use the first GPU
device = torch.device('cuda:1')  #Use second GPU

model.to(device)  #Move model to GPU
image = image.to(device)  #Move data to GPU

(2) Implementation by C ++

• When using CPU

CPU.cpp

torch::Device device(torch::kCPU);  //Use CPU

model->to(device);  //Move model to CPU
image = image.to(device);  //Move data to CPU

• When using GPU

GPU.cpp

torch::Device device(torch::kCUDA);     //Use default GPU
torch::Device device(torch::kCUDA, 0);  //Use the first GPU
torch::Device device(torch::kCUDA, 1);  //Use second GPU

model->to(device);  //Move model to GPU
image = image.to(device);  //Move data to GPU

2. Difference between deterministic and non-deterministic (GPU main operation & Python only)

In the Python version of PyTorch, in the case of learning using GPU, cuDNN is used to ** improve the learning speed **.

However, unlike C ++, just because the learning speed is improved does not mean that the exact same situation can be reproduced by turning the learning again.

Therefore, the PyTorch formula states that the behavior of cuDNN needs to be deterministic as follows in order to ensure reproducibility, and at the same time, the speed decreases.

https://pytorch.org/docs/stable/notes/randomness.html

Deterministic mode can have a performance impact, depending on your model. This means that due to the deterministic nature of the model, the processing speed (i.e. processed batch items per second) can be lower than when the model is non-deterministic.

From the engineer's point of view, we included it in this speed comparison because we may be concerned about the presence or absence of reproducibility and the speed changes depending on the presence or absence of reproducibility.

Unlike the "rand" function in C ++, if you do not set any initial value of the random number, it will be random, so in order to ensure reproducibility in Python, ** explicitly ** the initial value of the random number should be set. You need to set it. (The setting of the initial value of the random number does not affect the speed.)

The implementation is as follows.

• Deterministic case

deterministic.py

seed = 0
torch.manual_seed(seed)
torch.cuda.manual_seed(seed)
np.random.seed(seed)
random.seed(seed)
torch.backends.cudnn.deterministic = True  #Deterministic instead of slowing down
torch.backends.cudnn.benchmark = False     #Deterministic instead of slowing down

• Non-deterministic case

non_deterministic.py

torch.backends.cudnn.deterministic = False  #Faster instead of being non-deterministic
torch.backends.cudnn.benchmark = True       #Speed up when the image size does not change

3. Implementation differences between programming languages

Even if the content you want to implement is the same, if the programming language changes, ** notation and rules ** may change, and ** required libraries ** may change. Since both Python and C ++ are object-oriented languages, the concepts themselves are similar, but above all, Python is an interpreter type and C ++ is a compiled type, so it must be implemented considering that dynamic typing does not work for C ++. Also, since PyTorch's C ++ API is currently under development, it should be taken into consideration that some functions are not available.

Based on these points, I will introduce the implementation differences between Python and C ++, and the programs I implemented.

(1) Library usage

In addition to the usage status of the Python library that is generally written at present, the library recommended for implementation in C ++ and the usage status of the library of the program that I actually wrote are also described.

** At the moment (2020/03/24) </ font> **, it looks like the above.

using namespace torch;
namespace po = boost::program_options;

struct ConvolutionalAutoEncoderImpl : nn::Module{
private:
    nn::Sequential encoder, decoder;
public:
    ConvolutionalAutoEncoderImpl(po::Variables_map &vm);
    torch::Tensor forward(torch::Tensor x);
}

TORCH_MODULE(ConvolutionalAutoEncoder);

nn::Sequential sq;
sq->push_back(nn::Conv2d(nn::Conv2dOptions(3, 64, /*kernel_size=*/4).stride(2).padding(1).bias(false)));
sq->push_back(nn::BatchNorm2d(64));
sq->push_back(nn::ReLU(nn::ReLUOptions().inplace(true)));

void transforms::Normalize::forward(torch::Tensor &data_in, torch::Tensor &data_out){
    torch::Tensor data_out_src = (data_in - this->mean) / this->std;
    data_out = data_out_src.clone();
    return;
}