Deep Learning Java from scratch Chapter 2 Perceptron

table of contents

• Chapter 1 Introduction
• Chapter 2 Perceptron
• Chapter 3 Neural Network
• Chapter 4 Learning Neural Networks
• Chapter 5 Backpropagation Method
• Chapter 6 Learning Techniques
• 6.1 Parameter Update
• 6.2 Initial value of weight
  • 6.3 Batch Normalization
• 6.4 Regularization
• 6.5 Hyperparameter validation
• Chapter 7 Convolutional Neural Network
• Chapter 8 Deep Learning
• Finally

2.3 Implementation of Perceptron

2.3.1 Easy implementation

Perceptrons can be implemented as functions (static methods).

public static int AND(int x1, int x2) {
    double w1 = 0.5, w2 = 0.5, theta = 0.7;
    double tmp = x1 * w1 + x2 * w2;
    if (tmp <= theta)
        return 0;
    else
        return 1;
}

assertEquals(0, AND(0, 0));
assertEquals(0, AND(1, 0));
assertEquals(0, AND(0, 1));
assertEquals(1, AND(1, 1));

2.3.2 Introducing weights and biases

Using ND4J, it becomes as follows.

INDArray x = Nd4j.create(new double[] {0, 1});
INDArray w = Nd4j.create(new double[] {0.5, 0.5});
double b = -0.7;
assertEquals("[0.00,0.50]", Util.string(w.mul(x)));
assertEquals(0.5, w.mul(x).sumNumber().doubleValue(), 0.000005);
assertEquals(-0.19999999999999996, w.mul(x).sumNumber().doubleValue() + b, 0.000005);

2.3.3 Implementation by weight and bias

Using the "weight and bias method", AND, NDND, and OR gates can be implemented as follows.

public static int AND2(int x1, int x2) {
    INDArray x = Nd4j.create(new double[] {x1, x2});
    INDArray w = Nd4j.create(new double[] {0.5, 0.5});
    double b = -0.7;
    double tmp = w.mul(x).sumNumber().doubleValue() + b;
    return tmp <= 0 ? 0 : 1;
}

public static int NAND(int x1, int x2) {
    INDArray x = Nd4j.create(new double[] {x1, x2});
    INDArray w = Nd4j.create(new double[] {-0.5, -0.5});
    double b = 0.7;
    double tmp = w.mul(x).sumNumber().doubleValue() + b;
    return tmp <= 0 ? 0 : 1;
}

public static int OR(int x1, int x2) {
    INDArray x = Nd4j.create(new double [] {x1, x2});
    INDArray w = Nd4j.create(new double[] {0.5, 0.5});
    double b = -0.2;
    double tmp = w.mul(x).sumNumber().doubleValue() + b;
    return tmp <= 0 ? 0 : 1;
}

assertEquals(0, AND2(0, 0));
assertEquals(0, AND2(1, 0));
assertEquals(0, AND2(0, 1));
assertEquals(1, AND2(1, 1));
assertEquals(1, NAND(0, 0));
assertEquals(1, NAND(1, 0));
assertEquals(1, NAND(0, 1));
assertEquals(0, NAND(1, 1));
assertEquals(0, OR(0, 0));
assertEquals(1, OR(1, 0));
assertEquals(1, OR(0, 1));
assertEquals(1, OR(1, 1));

2.5 Multilayer Perceptron

2.5.2 Implementation of XOR gate

XOR gates can be mounted by using multiple layers of perceptrons.

public static int XOR(int x1, int x2) {
    int s1 = NAND(x1, x2);
    int s2 = OR(x1, x2);
    int y = AND2(s1, s2);
    return y;
}

assertEquals(0, XOR(0, 0));
assertEquals(1, XOR(1, 0));
assertEquals(1, XOR(0, 1));
assertEquals(0, XOR(1, 1));