I tried to implement Cifar10 with SONY Deep Learning library NNabla [Nippon Hurray]

It seems that SONY did it

Sony accelerates the evolution of AI. Open source deep learning core library

SONY has released its own Deep Learning library as open source. Official site Click here if you are interested

Isn't Japan doing its best, which is ridiculed as an IT underdeveloped country in some areas? In particular, major Japanese companies do not hear good rumors about software, so ...

Recently, such a blog about SONY has become a hot topic ... Retired from Sony Corporation

It's a big company, so I don't know if everything is as per this blog, I'm sure I didn't have a good impression. Personally, I don't really feel like supporting it because it's a Japanese company. Since it's a big deal, let's write an impression of using the library on Qiita!

Yes, everyone loves Cifar10

Click here for code

For the time being, I did it by defeating it. Yeah. Don't worry about the poor quality. ~~ I wanted to write an article about using NNabla first on Qiita, so this is the result of finishing it at super speed! !! !! ~~ Don't worry about the accuracy as you don't have time and the model is super suitable. .. I just wanted to touch the API. ..

main.py


def convolution(x):
  x = x.reshape([BATCH_SIZE, IMAGE_DEPTH, IMAGE_HEIGHT, IMAGE_WIDTH])
  with nn.parameter_scope("conv1"):
    output = PF.convolution(x, 16, (5, 5), stride=(2, 2), pad=(1, 1))
    output = F.relu(output)
  
  with nn.parameter_scope("conv2"):
    output = PF.convolution(output, 32, (3, 3), stride=(1, 1), pad=(1, 1))
    output = F.relu(output)
  
  with nn.parameter_scope("conv3"):
    output = PF.convolution(output, 64, (3, 3), stride=(1, 1), pad=(1, 1))
    output = F.relu(output)

  output = output.reshape([BATCH_SIZE, int(output.size / BATCH_SIZE)])

  with nn.parameter_scope("fc1"):
    output = PF.affine(output, 1024)
    output = F.relu(output)
  
  with nn.parameter_scope("fc2"):
    output = PF.affine(output, 256)
    output = F.relu(output)
  
  with nn.parameter_scope("softmax"):
    output = PF.affine(output, 10)
    output = F.softmax(output)
  
  return output

It's a CNN like this. The API reference Parametric Functions is an abstract API that makes it fairly easy to define a network. The low-layer one is this. (I haven't looked at it in detail ...)

Step: 00290 Train loss: 7.17657 Train accuracy: 0.34375
Step: 00300 Test loss: 7.22971 Test accuracy: 0.34990
Step: 00300 Train loss: 7.23585 Train accuracy: 0.28125
Step: 00310 Train loss: 7.26531 Train accuracy: 0.28125
Step: 00320 Train loss: 7.15616 Train accuracy: 0.37500
Step: 00330 Train loss: 7.19948 Train accuracy: 0.29688
Step: 00340 Train loss: 7.23404 Train accuracy: 0.26562
Step: 00350 Train loss: 7.13390 Train accuracy: 0.42188
Step: 00360 Train loss: 7.27805 Train accuracy: 0.20312
Step: 00370 Train loss: 7.08152 Train accuracy: 0.46875
Step: 00380 Train loss: 7.17094 Train accuracy: 0.34375
Step: 00390 Train loss: 7.12861 Train accuracy: 0.39062
Step: 00400 Test loss: 7.21064 Test accuracy: 0.36996

The execution result is output to the console like this. It feels lighter than writing a similar network in TensorFlow.

Difficulties

I usually only touch TensorFlow, and in fact I have never used Deep Learning libraries other than TF. .. It's embarrassing. I've used scratches to deepen my understanding.

To be honest, I don't have much difficulty. If you have used other libraries, can you use them without any discomfort? It's easy because you can define abstract layers. I can't say anything because I haven't used other libraries, but I could use this kind of code with the same feeling.

Speaking of the difficult points, it's been a long time since I used python2 system. I completely forgot the int division specification.

main.py


def accuracy(p, t):
  pred_and_label = [(np.argmax(_p), np.argmax(_t)) for _p, _t in zip(p.d, t.d)]
  return float(len(filter(lambda x: x[0] == x[1], pred_and_label))) / float(len(p.d))

I am calculating the correct answer rate of the output result here. This guy stayed at 0 all the time, and I was thinking for a while that it was strange. .. This has been fixed to float, so it will be output properly.

~~ Hmm. I mostly use python3 for business, so I want to support 3 systems as soon as possible here. ~~ It seems that it supports python3! Thank you for your comment! (2017/08/18)

** Addition **

Maybe I had a hard time, or maybe I just couldn't find it because I didn't have time, I couldn't find anything that corresponds to None in the shape definition of TensorFlow. The one that defines a shape like [?, 3, 64, 64]. Looking at the code on Github of Sample, it seems that the test data was calculated by repeating batches of the same size as the training, so I did that too. I'm not honest that the batch size can only be fixed, so I wonder if there is any way. .. Let's examine that area later. ~~ If you have time! !! ~~

Summary

~~ Personally, use TensorFlow ~~ I want the Deep Learning area to get excited, and I want you to do your best. If I have a free time, I think I should touch it more and pull it.

Also, I'm a little worried about this DynamicNN, so I'll touch it again when I have time. It sounds like a multi-paradigm of Define by run and Define and run.

Please let me know if there is something wrong! !!

Recommended Posts

I tried to implement Cifar10 with SONY Deep Learning library NNabla [Nippon Hurray]
I tried to implement deep learning that is not deep with only NumPy
[Python] Deep Learning: I tried to implement deep learning (DBN, SDA) without using a library.
I tried to implement Deep VQE
I tried to implement ListNet of rank learning with Chainer
I tried to implement Perceptron Part 1 [Deep Learning from scratch]
I tried to divide with a deep learning language model
I tried to implement Autoencoder with TensorFlow
I tried to make deep learning scalable with Spark × Keras × Docker
I tried to implement CVAE with PyTorch
[Deep Learning from scratch] I tried to implement sigmoid layer and Relu layer.
I tried to visualize the model with the low-code machine learning library "PyCaret"
I tried to implement reading Dataset with PyTorch
I tried deep learning
I tried to extract a line art from an image with Deep Learning
I tried to make deep learning scalable with Spark × Keras × Docker 2 Multi-host edition
I tried to move machine learning (ObjectDetection) with TouchDesigner
I tried to implement Minesweeper on terminal with python
I tried to implement an artificial perceptron with python
I tried to implement time series prediction with GBDT
[Introduction to Pytorch] I tried categorizing Cifar10 with VGG16 ♬
I tried to implement Grad-CAM with keras and tensorflow
[Deep Learning from scratch] I tried to explain Dropout
I tried to implement SSD with PyTorch now (Dataset)
"Deep Learning from scratch" Self-study memo (No. 16) I tried to build SimpleConvNet with Keras
"Deep Learning from scratch" Self-study memo (No. 17) I tried to build DeepConvNet with Keras
I tried to implement PCANet
I tried to implement StarGAN (1)
I tried to implement a volume moving average with Quantx
Mayungo's Python Learning Episode 3: I tried to print numbers with print
I tried to implement Harry Potter sort hat with CNN
I captured the Touhou Project with Deep Learning ... I wanted to.
I tried to implement SSD with PyTorch now (model edition)
I tried to make Othello AI that I learned 7.2 million hands by deep learning with Chainer
I tried machine learning with liblinear
I tried to implement adversarial validation
I tried to implement hierarchical clustering
I tried deep learning using Theano
I tried to implement Realness GAN
I tried learning LightGBM with Yellowbrick
I tried deep reinforcement learning (Double DQN) for tic-tac-toe with ChainerRL
I tried to implement sentence classification by Self Attention with PyTorch
I tried to implement PLSA in Python
I tried to implement permutation in Python
I tried to implement a blockchain that actually works with about 170 lines
I tried to visualize AutoEncoder with TensorFlow
I tried to get started with Hy
I tried to implement PLSA in Python 2
I tried learning with Kaggle's Titanic (kaggle②)
Mayungo's Python Learning Episode 2: I tried to put out characters with variables
[Reinforcement learning] Finally surpassed humans! ?? I tried to explain / implement Agent57 (Keras-RL)
I tried to implement ADALINE in Python
(Machine learning) I tried to understand Bayesian linear regression carefully with implementation.
I tried to implement PPO in Python
I tried to solve TSP with QAOA
I tried to classify Oba Hana and Emiri Otani by deep learning
I tried the common story of using Deep Learning to predict the Nikkei 225
I tried to implement merge sort in Python with as few lines as possible
I tried to extract named entities with the natural language processing library GiNZA
Mayungo's Python Learning Episode 5: I tried to do four arithmetic operations with numbers
I tried to implement various methods for machine learning (prediction model) using scikit-learn.