Deep Learning Java from scratch Chapter 1 Introduction

table of contents


["Deep Learning from scratch-the theory and implementation of deep learning learned in Python"] ( (written by O'Reilly Japan Yasuki Saito) is a book that explains Deep Learning from the beginning. I'm using Python as my programming language, but this post (and the series of posts that follow) will implement it in Java. However, [Deep Learning from scratch] It does not replace ( and is described on the assumption that it will be read together.

External library to use


[Deep Learning from scratch] ( uses NumPy as a library for numerical calculation. This post uses ND4J. ND4J is a library for Java that has similar functions to NumPy and Matlib. It is famous for being used in the open source Deeplearning4J. However, the documentation is not very well organized. It is necessary to learn how to use it by some trial and error. ND4J can use GPU via CUDA. I think it is advantageous for speeding up.


[Deep Learning from scratch] ( uses Matplotlib as a library for drawing graphs. Since graphs are not essential for deep learning, this post does not use a library for drawing graphs.

How to proceed

[Deep Learning from scratch] The programs written in Python that appear in ( will be rewritten in Java one by one. Write it as JUnit code to make sure the results are the same. For example, it looks like the following.

public class C1_5_NumPy {

    public void C1_5_2_Generating a NumPy array() {
        INDArray x = Nd4j.create(new double[] {1.0, 2.0, 3.0});
        assertEquals("[1.00,2.00,3.00]", Util.string(x));

This is a Java version of the sample program that appears in "1 Introduction to Python 1.5 NumPy 1.5.2 NumPy Array Generation". Since Japanese is used for the class name and method name, it may not work depending on the environment. It works fine in my environment (Windows10 + Eclipse Oxygen.2 Release 4.7.2). Since only the sections where the Python code is written are described, the heading item numbers are not serial numbers. All programs written in Java are posted on saka1029 / Deep.Learning on GitHub.

Environmental setting

In order to build a project on GitHub, you need to set the following dependencies in pom.xml.


You also need to set the project settings on the IDE to use * Java 8 *. With Java9, the classloader may throw an exception at runtime.

1.5 ND4J

[Deep Learning from scratch] ( describes Python, Numpy, and Matplotlib in Chapter 1, but this post only describes ND4J.

1.5.2 ND4J array generation

In ND4J, the array type is INDArray. To initialize by giving an initial value, use the factory method of Nd4j class as follows. Util.string (INDArray) is a string of array contents It is a self-made utility function to be converted.

INDArray x = Nd4j.create(new double[] {1.0, 2.0, 3.0});
assertEquals("[1.00,2.00,3.00]", Util.string(x));

1.5.3 Arithmetic calculation of ND4J

The four arithmetic operations of a one-dimensional array are as follows.

INDArray x = Nd4j.create(new double[] {1.0, 2.0, 3.0});
INDArray y = Nd4j.create(new double[] {2.0, 4.0, 6.0});
assertEquals("[3.00,6.00,9.00]", Util.string(x.add(y)));
assertEquals("[-1.00,-2.00,-3.00]", Util.string(x.sub(y)));
assertEquals("[2.00,8.00,18.00]", Util.string(x.mul(y)));
assertEquals("[0.50,0.50,0.50]", Util.string(x.div(y)));

INDArray .mul (INDArray) is multiplication by element, and matrix product is [INDArray](https: / / mmul (INDArray). INDArray also has add (Number), sub (Number), mul (Number), div Since (Number) is overloaded, the four arithmetic operations with scalar values can be performed in the same way. The entity of x is a two-dimensional array with 1 row and 3 columns. Note that this is different from NumPy. INDArray .rank () returns the dimensions of the array.

assertArrayEquals(new int[] {1,3}, x.shape());
assertEquals(2, x.rank());

1.5.4 N-dimensional array of ND4J

It is the same idea as NumPy in that it uses INDArray even if it becomes 2D or more. To create a two-dimensional array with initial values, do as follows.

INDArray A = Nd4j.create(new double[][] {{1, 2}, {3, 4}});
assertEquals("[[1.00,2.00],[3.00,4.00]]", Util.string(A));
assertArrayEquals(new int[] {2,2}, A.shape());
assertEquals(2, A.rank());

1.5.5 broadcast

With NumPy, it is possible to perform four arithmetic operations on arrays with different dimensions as they are, but with ND4J, that is not possible. You need to explicitly align the dimensions using the INDArray.broadcast (int []) method. .. If you forget this, an IllegalStateException will be thrown.

INDArray A = Nd4j.create(new double[][] {{1, 2}, {3, 4}});
INDArray B = Nd4j.create(new double[] {10, 20});
//INDArray method mul(INDArray)And add(INDArray)Does not broadcast automatically.
// broadcast(int[])You need to use to fit the left dimension.
assertEquals("[[10.00,40.00],[30.00,80.00]]", Util.string(A.mul(B.broadcast(A.shape()))));
//IllegalStateException when simply multiplied:It will be Mismatched shapes.
try {
    assertEquals("[[10.00,40.00],[30.00,80.00]]", Util.string(A.mul(B)));
} catch (IllegalStateException e) {
    assertEquals("Mis matched shapes", e.getMessage());
//Or mulRowVector(INDArray)You can also use.
assertEquals("[[10.00,40.00],[30.00,80.00]]", Util.string(A.mulRowVector(B)));

1.5.6 Access to elements

INDArray does not implement the Iterable interface. Also, there is no method that returns an Iterator. Access to the element must be done using subscripts. INDArray.getDouble (int ...) to retrieve elements, [INDArray] to retrieve rows ( (int), INDArray to retrieve columns Use org / nd4j / linalg / api / ndarray / INDArray.html) .getColumn (int). To change a 2D array to 1D, use the Nd4j.toFlattened (INDArray) method.

INDArray X = Nd4j.create(new double[][] {{51, 55}, {14, 19}, {0, 4}});
assertEquals("[[51.00,55.00],[14.00,19.00],[0.00,4.00]]", Util.string(X));
assertEquals("[51.00,55.00]", Util.string(X.getRow(0)));
assertEquals(55.0, X.getDouble(0, 1), 5e-6);
//INDArray does not implement the Iterable interface.
for (int i = 0, size = X.size(0); i < size; ++i)
    assertEquals(2, X.getRow(i).size(1));
//Convert X to a vector.
X = Nd4j.toFlattened(X);
assertEquals("[51.00,55.00,14.00,19.00,0.00,4.00]", Util.string(X));
//You can also retrieve all specified elements.
assertEquals("[51.00,14.00,0.00]", Util.string(X.getColumns(0, 2, 4)));

Data type

Looking at the examples so far, it looks like the INDArray holds an array of doubles internally because it is initialized using an array of doubles. However, ND4J reserves an array of floats by default. The documentation has the following description:

** Data type setting ** ND4J currently allows backing with INDArray with float or double precision values. The default is single precision (float). To configure ND4J to use double precision for the entire array, you can use: ** 0.4-rc3.8 and earlier, ** Nd4j.dtype = DataBuffer.Type.DOUBLE; NDArrayFactory factory = Nd4j.factory(); factory.setDType(DataBuffer.Type.DOUBLE); ** 0.4-rc3.9 and later, ** DataTypeUtil.setDTypeForContext(DataBuffer.Type.DOUBLE);

Let's create an array of doubles using DataTypeUtil.

//Create an array with default precision.
INDArray a = Nd4j.create(new double[] {1D / 3});
//Double precision(double)Set to.
//You can see that it has been changed to double.
assertEquals(DataBuffer.Type.DOUBLE, DataTypeUtil.getDtypeFromContext());
//Create a double precision array.
INDArray b = Nd4j.create(new double[] {1D / 3});
//a was initialized with double but is a single precision array.
assertEquals(0.3333333432674408, a.getDouble(0), 5e-14);
//b was initialized with double but is a double precision array.
assertEquals(0.3333333333333333, b.getDouble(0), 5e-14);
//Single precision(float)Return to.

I think that ND4J uses float by default mainly because it is intended for use in deep learning. ND4J can use GPU via CUDA, but float is probably easier to handle in that case as well. This series of posts consistently uses double type, but the inside of INDArray is float type. Please note that.

Recommended Posts

Deep Learning Java from scratch Chapter 1 Introduction
Deep Learning Java from scratch Chapter 2 Perceptron
Deep Learning from scratch Java Chapter 4 Neural network learning
Deep Learning Java from scratch Chapter 3 Neural networks
Deep Learning Java from scratch 6.4 Regularization
Study Deep Learning from scratch in Java.
Deep Learning Java from scratch 6.1 Parameter update
Deep Learning Java from scratch 6.3 Batch Normalization
Deep Learning Java from scratch Chapter 5 Error back propagation method
[Deep Learning from scratch] in Java 3. Neural network
Deep Learning Java from scratch 6.2 Initial values of weights
Java Performance Chapter 1 Introduction
Fastest PC setup for deep learning from scratch
[Deep Learning from scratch] 2. There is no such thing as NumPy in Java.
Java life starting from scratch
[Deep Learning from scratch] in Java 1. For the time being, differentiation and partial differentiation
Quick learning Java "Introduction?" Part 3 Talking away from programming
Java learning (0)
Java scratch scratch
[Java] Introduction
First steps for deep learning in Java
Note: next ・ nextLine (paiza learning Java introduction 9: # 06)
Learning for the first time java [Introduction]
Introduction to monitoring from Java Touching Prometheus
Effective Java Chapter 2
For JAVA learning (2018-03-16-01)
Quick learning Java "Introduction?" Part 1 Building an environment
Java learning day 5
I tried to implement deep learning in Java
Effective Java Chapter 6 34-35
[Java] Introduction to Java
Introduction to java
Effective Java Chapter 4 15-22
Effective Java Chapter 3
java learning day 2
java learning day 1
Build VS Code + WSL + Java + Gradle environment from scratch
Quick learning Java "Introduction?" Part 2 Let's write the process
[Note] Create a java environment from scratch with docker
Call Java from JRuby
Changes from Java 8 to Java 11
Sum from Java_1 to 100
Java learning 2 (learning calculation method)
java learning (conditional expression)
Eval Java source from Java
Java Learning (1)-Hello World
Rails Tutorial Chapter 3 Learning
JAVA learning history interface
Access API.AI from Java
Java learning memo (basic)
From Java to Ruby !!
Rails Tutorial Chapter 4 Learning
Java learning memo (interface)
Rails Tutorial Chapter 1 Learning
Rails Tutorial Chapter 2 Learning
Introduction to java command
Java learning memo (inheritance)
Java Performance Chapter 3 Java Performance Toolbox
Object-oriented child !? I tried Deep Learning in Java (trial edition)
Learning from the basics Artificial intelligence textbook Chapter 3 End-of-chapter problems
Learning from the basics Artificial intelligence textbook Chapter 4 Chapter end problems