[Deep Learning from scratch] in Java 1. For the time being, differentiation and partial differentiation


There is a book called "Deep Learning from scratch The theory and implementation of deep learning learned with Python". I read it twice, but I think I understand it, I don't understand it. Since it is implemented in Python in the first place, as a Java developer, I feel that it has been fooled. Because of the dynamic typing, the arguments of the same method are sometimes numbers and sometimes arrays, depending on what the caller passes ... too tricky ... ~~ What you should learn about Deeplearning4j obediently ~~ "Yes, let's implement it in Java". Please refer to the book for the explanation because it is only implemented.

For the time being, differentiation

Is it possible to implement differentiation and gradient in Java in the first place (P97 4.3 Numerical differentiation / P03 4.4 Gradient)? If I can't, it's unlikely that I'll be able to do anything, so I experimented for the time being (Java 8 or later).


private static double h = 1e-4; //Very small number
public double numericalDiff(DoubleUnaryOperator func, double x){
	return (func.applyAsDouble(x + h) - func.applyAsDouble(x-h))/ (2*h);

The content of the test is P103. It's just like the book, so it's considered good.


public void numericalDiff1(){
	assertThat(target.numericalDiff(p-> p*p+4*4, 3.0), is(6.00000000000378));
	assertThat(target.numericalDiff(p-> 3*3+p*p, 4.0), is(7.999999999999119));

Next, partial differential

Implemented P104 of the book. ~~ In the implementation of the book (Python), the original value is assigned to tmp_val, and after calculation, it is returned to the original value. However, if you do it in Java, the original data will change after all because the reference destination is the same. Therefore, a deep copy is used to hold the original data. ~~ → I received a comment that there is no problem if I calculate immediately after substitution. It's reasonable.


private static double h = 1e-4; //Very small number
public double[][] numericalGradient(ToDoubleFunction<double[][]> func, double[][] x){

	int cntRow = x.length;
	int cntCol = x[0].length;

	double[][] result = new double[cntRow][cntCol];
	for (int i=0; i < cntRow; i++){
		for (int j=0; j < cntCol; j++){

			double[][] xPlus = deepCopy(x);
			xPlus[i][j] = xPlus[i][j] + h;

			double[][] xMinus = deepCopy(x);
			xMinus[i][j] = xMinus[i][j] - h;

			result[i][j] = (func.applyAsDouble(xPlus) - func.applyAsDouble(xMinus))/ (2*h);

	return result;

public double[][] deepCopy(double[][] x){
	double[][] copy = new double[x.length][];
	for (int i = 0; i < copy.length; i++){
		copy[i] = new double[x[i].length];
		System.arraycopy(x[i], 0, copy[i], 0, x[i].length);
	return copy;

The content of the test is P104. Similarly, because it is as per the book, it is considered to be good.


public void numericalGradient(){

	ToDoubleFunction<double[][]> function = p-> p[0][0] * p[0][0] + p[0][1]*p[0][1];
	double[][] x = {{3,4}};
	double[][] result = target.numericalGradient(function, x);

	assertThat(result[0][0], is(6.00000000000378));
	assertThat(result[0][1], is(7.999999999999119));

	result = target.numericalGradient(function, new double[][]{{0,2}});

	assertThat(result[0][0], is(closeTo(0.0, 0.000001)));
	assertThat(result[0][1], is(closeTo(4.0, 0.000001)));

in conclusion

Differentiation and partial differentiation seem to be okay. By the way, I implemented all of them. The problem is that the PC is slow and I can not verify whether it is finally outputting proper results orz

Recommended Posts

[Deep Learning from scratch] in Java 1. For the time being, differentiation and partial differentiation
Study Deep Learning from scratch in Java.
[Deep Learning from scratch] in Java 3. Neural network
Deep Learning Java from scratch 6.4 Regularization
First steps for deep learning in Java
Deep Learning Java from scratch Chapter 1 Introduction
Deep Learning Java from scratch 6.1 Parameter update
Learning for the first time java [Introduction]
Deep Learning Java from scratch Chapter 2 Perceptron
Deep Learning Java from scratch 6.3 Batch Normalization
For the time being, run the war file delivered in Java with Docker
Deep Learning from scratch Java Chapter 4 Neural network learning
Fastest PC setup for deep learning from scratch
Use Java external library for the time being
Deep Learning Java from scratch Chapter 3 Neural networks
[Deep Learning from scratch] 2. There is no such thing as NumPy in Java.
Impressions and doubts about using java for the first time in Android Studio
Learning memo when learning Java for the first time (personal learning memo)
Deep Learning Java from scratch 6.2 Initial values of weights
Enter from docker-compose up for the time being, and learn Docker while learning the basic design of Web server (Nginx) ①
Deep Learning Java from scratch Chapter 5 Error back propagation method
JSON in Java and Jackson Part 1 Return JSON from the server
Correct the character code in Java and read from the URL
Access Web API on Android with Get and process Json (Java for the time being)
Java14 came out, so I tried record for the time being
[DL4J] Java deep learning for the first time (handwriting recognition using a fully connected neural network)
[Learning record] I got the current time in Ruby and output a different greeting for each time.
Java12 came out, so I tried the switch expression for the time being
Introduction to java for the first time # 2
[First Java] Make something that works with Intellij for the time being
[Java] for Each and sorted in Lambda
[Socket communication (Java)] Impressions of implementing Socket communication in practice for the first time
I translated the grammar of R and Java [Updated from time to time]
Programming for the first time in my life Java 1st Hello World
[Java] Get and display the date 10 days later using the Time API added from Java 8.
I want you to use Scala as Better Java for the time being
Parse the date and time string formatted by the C asctime function in Java
Regarding the transient modifier and serialization in Java
ChatWork4j for using the ChatWork API in Java
[Java] Set the time from the browser with jsoup
The story of learning Java in the first programming
Feel the passage of time even in Java
I tried to implement deep learning in Java
Capture and save from selenium installation in Java
Install Amazon Corretto (preview) for the time being
For JAVA learning (2018-03-16-01)
Write ABNF in Java and pass the email address
[For beginners] DI ~ The basics of DI and DI in Spring ~
Java language from the perspective of Kotlin and C #
A note for Initializing Fields in the Java tutorial
Try running Spring Cloud Config for the time being
Java classes and instances to understand in the figure
This and that for editing ini in Java. : inieditor-java
Command to try using Docker for the time being
Store in Java 2D map and turn with for statement
[For beginners] Explanation of classes, instances, and statics in Java
[Java] Make variables in extended for statement and for Each statement immutable
I translated [Clone method for Java arrays] as the Clone method in Java arrays.
Find the address class and address type from the IP address with Java
Object-oriented child !? I tried Deep Learning in Java (trial edition)
Hello World with Ruby extension library for the time being