numpy-sigmoid learning

numpy-sigmoid learning

A reminder of self-learning. Arrange diagrams and learning code to understand numpy-sigmoid

** When the sigmoid function is used as the activation function, it becomes as follows **

python

import numpy as np


#Definition of sigmoid function
def sigmoid(a):
    return 1 / (1 + np.exp(-a))


#First layer
x1 = np.array([[1], [2]])
w1 = np.array([[0.4, 0.3], [0.7, 0.6]])
a1 = w1.dot(x1)

#Second layer
x2 = sigmoid(a1)
w2 = np.array([[0.2, 0.3], [1, 0.7]])
a2 = w2.dot(x2)

#Output value and correct answer value
y = sigmoid(a2)
t = np.array([[1], [0]])
print(y)

#Arrange input values, parameters, and total values
X = [x1, x2]
A = [a1, a2]
W = [w1, w2]

#Get the size of how many layers the parameter has
max_layer = len(X)


#Differentiation of activation function
def f(a):
    return (1 - sigmoid(a)) * sigmoid(a)


#Implementation of update formula g
def g(l, j):
    if max_layer == l:
        return (y[j] - t[j]) * f(A[l - 1][j])
    else:
        output = 0
        m = A[l - 1].shape[0]
        for i in range(m):
            output += g(l + 1, i) * W[l][j, i] * f(A[l - 1][j])
        return output


#Differentiation of error function by parameter w
def diff(j, k, l):
    return g(l, j) * X[l - 1][k]


#Learn parameters 100 times
for _ in range(100):
    for l in range(len(X)):
        for j in range(W[l].shape[0]):
            for k in range(W[l].shape[1]):
                W[l][j, k] = W[l][j, k] - diff(j, k, l + 1)
    A[0] = W[0].dot(X[0])
    X[1] = sigmoid(A[0])
    A[1] = W[1].dot(X[1])
    y = sigmoid(A[1])

print(y)

[[0.60041131] [0.79248709]] [[0.91175343] [0.08878309]]

Examination of shape

python

#range.Understand shape shape seems to return rows and columns
import numpy as np
a = np.array([[1,2,3],[4,5,6],[4,5,6],[4,5,6]])
a.shape

(4, 3)

In short, it's the size of the array. This result indicates that it is an array of 4 * 3 (4 rows and 3 columns). a.shape [0] is 4 in this example. 3 for a.shape [1]