DEEP PROBABILISTIC PROGRAMMING --- "Deep Learning + Bayes" Library --- Introduction of Edward

WHY

Edward

We will introduce it from the following viewpoints with expectations for mounting and future growth.

--Co-authored by Google Research --BackEnd is Tensorflow --There is something like Caffee's Model Zoo and there is an ecosystem for published models ――It converges faster than deep learning and is likely to become the next trend

I have read Deep Probabilistic Programming, so I will introduce an excerpt from this content.


What thing?

Introducing a library that is more flexible than conventional deep learning and enables probabilistic programming with high computational efficiency. The same model can be used with different inference machines. Since the model representation can be reused as part of inference, a wide variety of network configurations are possible. The backend is made of Tensorflow and there is something like Caffee's Model Zoo Probability Zoo


What is amazing compared to previous research?

・ Computational efficiency is better than deep learning ・ Creating flexible inference parts ・ 35 times faster with logistic regression than Stan and PyMC3


Where is the key to technology and method?

・ Consists of random value generation and inference ・ Created with open technology (backend is Tensorflow) -The part that includes the construction of the model in the inference part is different from the conventional one. ・ Bayesian Recurrent Neural Network with Variable Length


Concrete example

Bayesian Recurrent Neural Network with Variable Length

Normal: normal distribution

Learn the mean and variance of the normal distribution of Weight and regularization

Screen Shot 2017-02-28 at 8.01.05.png


GAN

A network for generation and a network for identification are configured, and data parameters are generated from the distribution. Predicted values are also generated from the distribution

Screen Shot 2017-02-28 at 8.03.09.png


Speculator composite

EM Algorithm

Screen Shot 2017-02-28 at 8.06.14.png


How did you verify it was valid?

Comparison of each method

Screen Shot 2017-02-28 at 8.07.27.png

Comparison with conventional library

Speed verification with PyMC and Stan

About 20 times that of Stan About 40 times that of PyMC

Spec
12-core Intel i7-5930K CPU 3.5GHz NVIDIA Titan X(Maxwell) GPU

Task
generate posterior samples with Hamiltonian Monte Carlo

Data
Covertype dataset (N = 581012, D = 54; responses were binarized)

Other
100 HMC iterations, with 10 leapfrog updates per iteration and a step size of 0.5/N

35x speedup from stan 

Is there a discussion?

The latest technology wants to incorporate more traditional Bayesian techniques I want to be able to apply it even to huge data


What paper should I read next?

The following paper is recommended because you can see how to use it in earnest. Tran, Dustin, et al. "Edward: A library for probabilistic modeling, inference, and criticism." arXiv preprint arXiv:1610.09787 (2016).


Reference

Deep Probabilistic Programming

Recommended Posts

DEEP PROBABILISTIC PROGRAMMING --- "Deep Learning + Bayes" Library --- Introduction of Edward
Python & Machine Learning Study Memo ②: Introduction of Library
Deep learning 1 Practice of deep learning
DNN (Deep Learning) Library: Comparison of chainer and TensorFlow (1)
Deep Learning Gaiden ~ GPU Programming ~
Deep running 2 Tuning of deep learning
Deep Reinforcement Learning 1 Introduction to Reinforcement Learning
1st month of programming learning
Deep reinforcement learning 2 Implementation of reinforcement learning
Introduction to Deep Learning ~ Backpropagation ~
Deep Learning Model Lightening Library Distiller
Introduction to Deep Learning ~ Function Approximation ~
Introduction to Deep Learning ~ Coding Preparation ~
Introduction to Machine Learning Library SHOGUN
Microsoft's Deep Learning Library "CNTK" Tutorial
Introduction to Deep Learning ~ Dropout Edition ~
Introduction to Deep Learning ~ Forward Propagation ~
Introduction to Deep Learning ~ CNN Experiment ~
Learning history of programming transcendence beginners
(python) Deep Learning Library Chainer Basics Basics
I tried using the trained model VGG16 of the deep learning library Keras
Othello-From the tic-tac-toe of "Implementation Deep Learning" (3)
Introduction to Deep Learning ~ Convolution and Pooling ~
Meaning of deep learning models and parameters
Try deep learning of genomics with Kipoi
Visualize the effects of deep learning / regularization
Introduction of Go's RDB access library (go-pg/pg)
Sentiment analysis of tweets with deep learning
Learning record of reading "Deep Learning from scratch"
Othello-From the tic-tac-toe of "Implementation Deep Learning" (2)
Free version of DataRobot! ?? Introduction to "PyCaret", a library that automates machine learning
Deep Learning
Introduction of new voice feature extraction library Surfboard
The story of doing deep learning with TPU
An amateur tried Deep Learning using Caffe (Introduction)
Deep learning / error back propagation of sigmoid function
A memorandum of studying and implementing deep learning
Basic understanding of stereo depth estimation (Deep Learning)
Introduction of Python Imaging Library (PIL) using HomeBrew
Parallel learning of deep learning by Keras and Kubernetes
Introduction to Deep Learning ~ Localization and Loss Function ~
[Translation] scikit-learn 0.18 Tutorial Introduction of machine learning by scikit-learn