["Deep Learning from scratch-the theory and implementation of deep learning learned in Python"] (https://www.oreilly.co.jp/books/9784873117584/) (written by O'Reilly Japan Yasuki Saito) is a book that explains Deep Learning from the beginning. I'm using Python as my programming language, but this post (and the series of posts that follow) will implement it in Java. However, [Deep Learning from scratch] It does not replace (https://www.oreilly.co.jp/books/9784873117584/) and is described on the assumption that it will be read together.
Numpy
[Deep Learning from scratch] (https://www.oreilly.co.jp/books/9784873117584/) uses NumPy as a library for numerical calculation. This post uses ND4J. ND4J is a library for Java that has similar functions to NumPy and Matlib. It is famous for being used in the open source Deeplearning4J. However, the documentation is not very well organized. It is necessary to learn how to use it by some trial and error. ND4J can use GPU via CUDA. I think it is advantageous for speeding up.
Matplotlib
[Deep Learning from scratch] (https://www.oreilly.co.jp/books/9784873117584/) uses Matplotlib as a library for drawing graphs. Since graphs are not essential for deep learning, this post does not use a library for drawing graphs.
[Deep Learning from scratch] The programs written in Python that appear in (https://www.oreilly.co.jp/books/9784873117584/) will be rewritten in Java one by one. Write it as JUnit code to make sure the results are the same. For example, it looks like the following.
public class C1_5_NumPy {
@Test
public void C1_5_2_Generating a NumPy array() {
INDArray x = Nd4j.create(new double[] {1.0, 2.0, 3.0});
assertEquals("[1.00,2.00,3.00]", Util.string(x));
}
}
This is a Java version of the sample program that appears in "1 Introduction to Python 1.5 NumPy 1.5.2 NumPy Array Generation". Since Japanese is used for the class name and method name, it may not work depending on the environment. It works fine in my environment (Windows10 + Eclipse Oxygen.2 Release 4.7.2). Since only the sections where the Python code is written are described, the heading item numbers are not serial numbers. All programs written in Java are posted on saka1029 / Deep.Learning on GitHub.
In order to build a project on GitHub, you need to set the following dependencies in pom.xml.
<dependencies>
<dependency>
<groupId>org.nd4j</groupId>
<artifactId>nd4j-native-platform</artifactId>
<version>0.9.1</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.2</version>
</dependency>
</dependencies>
You also need to set the project settings on the IDE to use * Java 8 *. With Java9, the classloader may throw an exception at runtime.
1.5 ND4J
[Deep Learning from scratch] (https://www.oreilly.co.jp/books/9784873117584/) describes Python, Numpy, and Matplotlib in Chapter 1, but this post only describes ND4J.
In ND4J, the array type is INDArray. To initialize by giving an initial value, use the factory method of Nd4j class as follows. Util.string (INDArray) is a string of array contents It is a self-made utility function to be converted.
INDArray x = Nd4j.create(new double[] {1.0, 2.0, 3.0});
assertEquals("[1.00,2.00,3.00]", Util.string(x));
The four arithmetic operations of a one-dimensional array are as follows.
INDArray x = Nd4j.create(new double[] {1.0, 2.0, 3.0});
INDArray y = Nd4j.create(new double[] {2.0, 4.0, 6.0});
assertEquals("[3.00,6.00,9.00]", Util.string(x.add(y)));
assertEquals("[-1.00,-2.00,-3.00]", Util.string(x.sub(y)));
assertEquals("[2.00,8.00,18.00]", Util.string(x.mul(y)));
assertEquals("[0.50,0.50,0.50]", Util.string(x.div(y)));
INDArray .mul (INDArray) is multiplication by element, and matrix product is [INDArray](https: / /nd4j.org/doc/org/nd4j/linalg/api/ndarray/INDArray.html). mmul (INDArray). INDArray also has add (Number), sub (Number), mul (Number), div Since (Number) is overloaded, the four arithmetic operations with scalar values can be performed in the same way. The entity of x is a two-dimensional array with 1 row and 3 columns. Note that this is different from NumPy. INDArray .rank () returns the dimensions of the array.
assertArrayEquals(new int[] {1,3}, x.shape());
assertEquals(2, x.rank());
It is the same idea as NumPy in that it uses INDArray even if it becomes 2D or more. To create a two-dimensional array with initial values, do as follows.
INDArray A = Nd4j.create(new double[][] {{1, 2}, {3, 4}});
assertEquals("[[1.00,2.00],[3.00,4.00]]", Util.string(A));
assertArrayEquals(new int[] {2,2}, A.shape());
assertEquals(2, A.rank());
With NumPy, it is possible to perform four arithmetic operations on arrays with different dimensions as they are, but with ND4J, that is not possible. You need to explicitly align the dimensions using the INDArray.broadcast (int []) method. .. If you forget this, an IllegalStateException will be thrown.
INDArray A = Nd4j.create(new double[][] {{1, 2}, {3, 4}});
INDArray B = Nd4j.create(new double[] {10, 20});
//INDArray method mul(INDArray)And add(INDArray)Does not broadcast automatically.
// broadcast(int[])You need to use to fit the left dimension.
assertEquals("[[10.00,40.00],[30.00,80.00]]", Util.string(A.mul(B.broadcast(A.shape()))));
//IllegalStateException when simply multiplied:It will be Mismatched shapes.
try {
assertEquals("[[10.00,40.00],[30.00,80.00]]", Util.string(A.mul(B)));
fail();
} catch (IllegalStateException e) {
assertEquals("Mis matched shapes", e.getMessage());
}
//Or mulRowVector(INDArray)You can also use.
assertEquals("[[10.00,40.00],[30.00,80.00]]", Util.string(A.mulRowVector(B)));
INDArray does not implement the Iterable interface. Also, there is no method that returns an Iterator. Access to the element must be done using subscripts. INDArray.getDouble (int ...) to retrieve elements, [INDArray] to retrieve rows (https://nd4j.org/doc/org/nd4j/linalg/api/ndarray/INDArray.html).getRow (int), INDArray to retrieve columns Use org / nd4j / linalg / api / ndarray / INDArray.html) .getColumn (int). To change a 2D array to 1D, use the Nd4j.toFlattened (INDArray) method.
INDArray X = Nd4j.create(new double[][] {{51, 55}, {14, 19}, {0, 4}});
assertEquals("[[51.00,55.00],[14.00,19.00],[0.00,4.00]]", Util.string(X));
assertEquals("[51.00,55.00]", Util.string(X.getRow(0)));
assertEquals(55.0, X.getDouble(0, 1), 5e-6);
//INDArray does not implement the Iterable interface.
for (int i = 0, size = X.size(0); i < size; ++i)
assertEquals(2, X.getRow(i).size(1));
//Convert X to a vector.
X = Nd4j.toFlattened(X);
assertEquals("[51.00,55.00,14.00,19.00,0.00,4.00]", Util.string(X));
//You can also retrieve all specified elements.
assertEquals("[51.00,14.00,0.00]", Util.string(X.getColumns(0, 2, 4)));
Looking at the examples so far, it looks like the INDArray holds an array of doubles internally because it is initialized using an array of doubles. However, ND4J reserves an array of floats by default. The documentation has the following description:
** Data type setting ** ND4J currently allows backing with INDArray with float or double precision values. The default is single precision (float). To configure ND4J to use double precision for the entire array, you can use: ** 0.4-rc3.8 and earlier, ** Nd4j.dtype = DataBuffer.Type.DOUBLE; NDArrayFactory factory = Nd4j.factory(); factory.setDType(DataBuffer.Type.DOUBLE); ** 0.4-rc3.9 and later, ** DataTypeUtil.setDTypeForContext(DataBuffer.Type.DOUBLE);
Let's create an array of doubles using DataTypeUtil.
//Create an array with default precision.
INDArray a = Nd4j.create(new double[] {1D / 3});
//Double precision(double)Set to.
DataTypeUtil.setDTypeForContext(DataBuffer.Type.DOUBLE);
//You can see that it has been changed to double.
assertEquals(DataBuffer.Type.DOUBLE, DataTypeUtil.getDtypeFromContext());
//Create a double precision array.
INDArray b = Nd4j.create(new double[] {1D / 3});
//a was initialized with double but is a single precision array.
assertEquals(0.3333333432674408, a.getDouble(0), 5e-14);
//b was initialized with double but is a double precision array.
assertEquals(0.3333333333333333, b.getDouble(0), 5e-14);
//Single precision(float)Return to.
DataTypeUtil.setDTypeForContext(DataBuffer.Type.FLOAT);
I think that ND4J uses float by default mainly because it is intended for use in deep learning. ND4J can use GPU via CUDA, but float is probably easier to handle in that case as well. This series of posts consistently uses double type, but the inside of INDArray is float type. Please note that.
Recommended Posts