Introduction to Thano Function definition and automatic differentiation

This is the article on the 6th day of Hiroshima University IT Engineer Advent Calendar 2019.

I'm Yuto Araki, a 4th year student at the Faculty of Engineering, Hiroshima University! I will be an IT engineer in Fukuoka from next year! I'm doing Twitter with the handle name Imikoto, so please check that as well. (https://twitter.com/es__135 )

Getting Started with Thano

python


import numpy as np
import theano.tensor as T
from theano import function

x = T.dscalar('x')
y = T.dscalar('y')
z = x + y
f = function([x,y],z)
f(2,3) #array(5.)Is output.

(Quoted from http://deeplearning.net/software/theano/tutorial/adding.html#exercise)

Define the variables used in the function below on the 5th and 6th lines, and create the function on the 7th and 8th lines. It may seem a little strange before you get used to it, but once you get used to it, it will become easier to write.

Let's write the three-square theorem by applying this.

x = T.dscalar('x')
y = T.dscalar('y')
Pythagoras = ( x**2 + y**2 )**(1/2)
length     = function([x,y],Pythagoras)
length(3,4) #5.0 is output.

Automatic differentiation

For more information, read Wikipedia, but you can calculate the derivative of the defined function.

x = T.dscalar('x')

#y = sin(x) + cos(x)
y = T.sin(x) + T.cos(x)

#The meaning of differentiating y with respect to x
gy = T.grad(cost=y, wrt=x)

f = function(inputs=[x], outputs=gy)

#Find the derivative by giving a concrete x
print (f(0))        #1.0
print (f(np.pi / 4))#1.1102230246251565e-16
print (f(np.pi))    #-1.0000000000000002

(This article was used as a reference. I recommend this article because it introduces many other examples.)

Also,

x = T.dscalar('x')
y = T.dscalar('y')
z =  T.sin(x) + T.cos(y)
#z = sin(x) + cos(y)

#The meaning of differentiating y with respect to x
dx    = T.grad(cost=z, wrt=x)
dy    = T.grad(cost=z, wrt=y)
dz_dx = function(inputs=[x,y], outputs=dx)
dz_dy = function(inputs=[x,y], outputs=dy)
#Find the derivative by giving a concrete x
pi = math.pi
dz_dx(pi/2,pi/2) #array(6.123234e-17)
dz_dy(0,pi/2)    #array(-1.)

In this way, you can also calculate the derivative of two variables. Could you somehow understand how to use it?

Summary

It's a little special way of writing, but once I got used to it, I thought it was surprisingly easy to write and use. If I have time, I would like to write an article about implementing neural networks using Thano. This is the library used in the early Deep Learning, and I can't imagine how it has evolved into a modern library. Will study

References

theano documentation http://deeplearning.net/software/theano/tutorial/adding.html#exercise

A record of artificial intelligence http://aidiary.hatenablog.com/entry/20150518/1431954329

github https://github.com/Theano/Theano

Recommended Posts

Introduction to Thano Function definition and automatic differentiation
Introduction to PyTorch (1) Automatic differentiation
[Introduction to Udemy Python3 + Application] 48. Function definition
Introduction to Deep Learning ~ Localization and Loss Function ~
[Introduction to Python3 Day 1] Programming and Python
Introduction and implementation of activation function
Introduction to Deep Learning ~ Function Approximation ~
[Introduction to Udemy Python3 + Application] 49. Function citation and return value declaration
Introduction to Deep Learning ~ Convolution and Pooling ~
[Introduction to Udemy Python3 + Application] 45. enumerate function
[Introduction to Udemy Python3 + Application] 41. Input function
[Introduction to Udemy Python3 + Application] 44. range function
[Introduction to Udemy Python3 + Application] 46. Zip function
zipline Function to buy and sell stocks
[Introduction to AWS] Text-Voice conversion and playing ♪
[Introduction to Python3 Day 12] Chapter 6 Objects and Classes (6.3-6.15)
[Introduction to Python3 Day 22] Chapter 11 Concurrency and Networking (11.1 to 11.3)
[Introduction to Udemy Python3 + Application] 64. Namespace and Scope
[Introduction to Python3 Day 11] Chapter 6 Objects and Classes (6.1-6.2)
[Python] How to use hash function and tuple.
[Introduction to PID] I tried to control and play ♬
Introduction to MQTT (Introduction)
Introduction to Scrapy (1)
Introduction to Scrapy (3)
Introduction to Supervisor
Introduction to Tkinter 1: Introduction
Introduction to PyQt
Introduction to Scrapy (2)
[Linux] Introduction to Linux
Introduction to Scrapy (4)
Introduction to discord.py (2)
Introduction to discord.py
[Introduction to Udemy Python3 + Application] 35. Comparison operators and logical operators
[Introduction to Python] How to iterate with the range function?
How to define Decorator and Decomaker in one function
Introduce pipe operators and function synthesis to Python (provisional)
Try MXNet Tutorial (2): Symbol-Neural Net Graph and Automatic Differentiation
[Introduction to Style GAN] Mayu Yu and anime smiled ♬
[Introduction to Udemy Python3 + Application] 68. Import statement and AS