Microsoft's Deep Learning Library "CNTK" Tutorial

Do you know CNTK? CNTK is an open source Deep Learning library led by Microsoft. To be honest, I recently started using it, but this CNTK is not like Microsoft in a good way, and it is quite good. I think it's cool. This time, I will first briefly introduce the features of CNTK, [I will look at a code example using Jupyter Notebook in the second half](http://qiita.com/usuyama/items/bb7600ab7cfc1f5b8be3#python%E3%81%A7cntk] % E3% 82% 92% E4% BD% BF% E3% 81% A3% E3% 81% A6% E3% 81% BF% E3% 82% 8B).

** [Tutorial here](http://qiita.com/usuyama/items/bb7600ab7cfc1f5b8be3#python%E3%81%A7cntk%E3%82%92%E4%BD%BF%E3%81%A3%E3% 81% A6% E3% 81% BF% E3% 82% 8B) **

cntk.png

Open development policy that does not stick to "Microsoft"

Open Source

https://github.com/Microsoft/CNTK

Development is underway on GitHub. There is no in-house version.

Official wiki

Python and C ++ API

There is no .Net API (yet). Brainscript is also supported.

Available on Linux and Windows

"Both Linux and Windows are first citizens of CNTK," the development team said. It seems that both platforms support the same functionality, although there may be a slight time lag. Docker is also available for Linux, so maybe Linux is easier to use. not.

ms_loves_linux.png

High speed & scale

Scale on multiple GPUs

This is a performance comparison with Therano, TensorFlow, Torch 7, and Caffe. This is a comparison of how many frames can be processed per second by a fully connected 4-layer neural network (minibatch size is 8192). The blue and orange settings are clusters of Linux machines with one or four GPUs (Nvidia K40), and green is a cluster of two Linux machines (2 x 4 GPUs). ** With one GPU, each library has almost the same speed (CNTK is faster than TensorFlow?), But only CNTK can effectively utilize multiple GPUs. ** With 4 GPUs, CNTK is almost quadrupled, but Torch 7 and Caffe are about doubling. TensorFlow is almost unchanged.

speed-comparison.PNG Microsoft Research Blog

October 2016: CNTK reaches human level in speech recognition accuracy

In an experiment in October 2016, a speech recognition engine developed by Microsoft's team recorded a word error rate of 5.9%. This was equivalent to asking a professional transcription service for the same audio file. What's more, this transcription service uses a two-step process, with the first person transcribing from 1 and the second person making corrections.

The detailed part of the method is published in Paper, but it uses a neural network with a very large number of layers. Again, it is stated in the Paper that CNTK was demonstrating its power and that high-speed processing of CNTK was essential.

speech-graphic.jpg

Try using CNTK in Python

Introduction to CNTK (Python)

My colleague made a good tutorial, so I translated it. Since it is a Jupyter Notebook, you can edit and execute the code on the spot without preparing the environment. We plan to add more in sequence.

-CNTK 101: Logistic Regression and Introduction to Machine Learning ――First, let's create a linear model and solve a simple two-class classification problem. Introducing the general flow from model construction, learning to evaluation. Not only does it follow the code, but it also briefly introduces the mathematics behind it. -CNTK 102: Learning simulation data with Feed Forward Network --Create a 4-layer neural network and learn the same simulation data as 101. - feedforward_network.jpg -CNTK 103 Part A: MNIST (handwritten number) data preparation --In Part A, you will download the MNIST data, convert it to an easy-to-use format, and save it in a file. - mnist_originals.png

-CNTK 103 Part B: Learn MNIST data with Feed Forward Network --Learning the MNIST data downloaded in Part A with the 4-layer neural network created in 102.

Jupyter Notebook (Azure Notebook) If you log in with your Microsoft account, you will be able to clone your notes for editing and execution. azure_notebook.png

It is very convenient to be able to change the code on the spot and execute it, so please try changing the code on the tutorial and playing with it.

jupyter_run_cell.png

reference

--If you want to prepare the environment locally, Official or [here](http://qiita.com/ The article of hiiyan0402 / items / ec41908f2aa81b73047e) seems to be helpful. -(I haven't tried it yet) Prepare the environment on Azure seems to be easier. You can also use GPU.

Recommended Posts

Microsoft's Deep Learning Library "CNTK" Tutorial
Deep Learning Model Lightening Library Distiller
Deep Learning
(python) Deep Learning Library Chainer Basics Basics
Deep learning tutorial from environment construction
Deep Learning Memorandum
Start Deep learning
Python Deep Learning
Deep learning × Python
Machine learning tutorial summary
Why what? Deep Learning Scientific Calculation Library Numpy Edition
Python: Deep Learning Practices
Deep learning / activation functions
Deep Learning from scratch
Deep learning 1 Practice of deep learning
Deep learning / cross entropy
First Deep Learning ~ Preparation ~
First Deep Learning ~ Solution ~
[AI] Deep Metric Learning
I tried deep learning
DNN (Deep Learning) Library: Comparison of chainer and TensorFlow (1)
Python: Deep Learning Tuning
Deep learning large-scale technology
Machine learning library Shogun
DEEP PROBABILISTIC PROGRAMMING --- "Deep Learning + Bayes" Library --- Introduction of Edward
Deep learning / softmax function
Sine wave prediction using RNN in deep learning library Keras
"Deep Learning from scratch" self-study memo (No. 15) TensorFlow beginner tutorial
Deep Learning from scratch 1-3 chapters
Try deep learning with TensorFlow
Deep Learning Gaiden ~ GPU Programming ~
<Course> Deep Learning: Day2 CNN
Deep learning image recognition 1 theory
Deep running 2 Tuning of deep learning
Deep learning / LSTM scratch code
Rabbit Challenge Deep Learning 1Day
<Course> Deep Learning: Day1 NN
Deep Kernel Learning with Pyro
Try Deep Learning with FPGA
Introducing Udacity Deep Learning Nanodegree
Introduction to Deep Learning ~ Learning Rules ~
Rabbit Challenge Deep Learning 2Day
Deep Reinforcement Learning 1 Introduction to Reinforcement Learning
Deep reinforcement learning 2 Implementation of reinforcement learning
Generate Pokemon with Deep Learning
Introduction to Deep Learning ~ Backpropagation ~
[Windows] Library Keras course where you can try Deep Learning immediately-Part 1