Deep learning from scratch (cost calculation)

This is a continuation of Deep Learning from Zero (Forward Propagation).

Last time, we implemented forward propagation to derive the probability $ AL $ that the input value is $ 1 $. This time, in order to improve the accuracy of the model, we will implement up to the point where the error between the predicted value and the correct label is derived.

cost

Since this DNN predicts whether the input image is a cat or not, we use the Cross-entropy error for the cost function.

Cross-entropy error

It is expressed by the following formula. $ AL $ is a prediction of the probability that Y is 1, and $ Y $ is the correct label. $L(AL, Y) = \sum^m_{i=1}{(-Ylog(AL) - (1-Y)log(1-A))}$

Returns $ -log (AL) $ when $ Y = 1 $ and $ log (1-AL) $ when $ Y = 0 $. From the graph below, you can see that the farther the predicted value and the correct answer are, the higher the cost.

cross_entropy_cost_y_0.png

cross_entropy_cost_y_1.png

The function is as follows.

def compute_cost(AL, Y):
    m = Y.shape[1]
    logprobs = np.multiply(Y, np.log(AL)) + np.multiply(1-Y, np.log(1-AL))
    cost = -1/m * np.sum(logprobs)
    return cost

Returns the cost by passing the vectorized predicted value $ AL $ and the correct label $ Y $.

Recommended Posts

Deep learning from scratch (cost calculation)
Deep Learning from scratch
Deep Learning from scratch 1-3 chapters
Deep Learning memos made from scratch
[Learning memo] Deep Learning made from scratch [Chapter 7]
Deep learning from scratch (forward propagation edition)
Deep learning / Deep learning from scratch 2-Try moving GRU
Deep learning / Deep learning made from scratch Chapter 6 Memo
[Learning memo] Deep Learning made from scratch [Chapter 5]
[Learning memo] Deep Learning made from scratch [Chapter 6]
"Deep Learning from scratch" in Haskell (unfinished)
Deep learning / Deep learning made from scratch Chapter 7 Memo
[Windows 10] "Deep Learning from scratch" environment construction
Learning record of reading "Deep Learning from scratch"
[Deep Learning from scratch] About hyperparameter optimization
"Deep Learning from scratch" Self-study memo (Part 12) Deep learning
[Learning memo] Deep Learning made from scratch [~ Chapter 4]
"Deep Learning from scratch" self-study memo (unreadable glossary)
"Deep Learning from scratch" Self-study memo (9) MultiLayerNet class
Good book "Deep Learning from scratch" on GitHub
Deep Learning from scratch Chapter 2 Perceptron (reading memo)
[Learning memo] Deep Learning from scratch ~ Implementation of Dropout ~
Python vs Ruby "Deep Learning from scratch" Summary
"Deep Learning from scratch" Self-study memo (10) MultiLayerNet class
"Deep Learning from scratch" Self-study memo (No. 11) CNN
Deep learning / LSTM scratch code
"Deep Learning from scratch" Self-study memo (No. 19) Data Augmentation
"Deep Learning from scratch 2" Self-study memo (No. 21) Chapters 3 and 4
Application of Deep Learning 2 made from scratch Spam filter
[Deep Learning from scratch] I tried to explain Dropout
Deep Learning / Deep Learning from Zero 2 Chapter 4 Memo
[Deep Learning from scratch] Implementation of Momentum method and AdaGrad method
An amateur stumbled in Deep Learning from scratch Note: Chapter 1
Deep Learning / Deep Learning from Zero Chapter 3 Memo
Deep Learning / Deep Learning from Zero 2 Chapter 5 Memo
Making from scratch Deep Learning ❷ An amateur stumbled Note: Chapter 5
Making from scratch Deep Learning ❷ An amateur stumbled Note: Chapter 2
An amateur stumbled in Deep Learning from scratch Note: Chapter 3
An amateur stumbled in Deep Learning from scratch Note: Chapter 7
An amateur stumbled in Deep Learning from scratch Note: Chapter 5
Making from scratch Deep Learning ❷ An amateur stumbled Note: Chapter 7
Making from scratch Deep Learning ❷ An amateur stumbled Note: Chapter 1
Making from scratch Deep Learning ❷ An amateur stumbled Note: Chapter 4
"Deep Learning from scratch" self-study memo (No. 18) One! Meow! Grad-CAM!
Deep Learning / Deep Learning from Zero 2 Chapter 7 Memo
Deep Learning / Deep Learning from Zero 2 Chapter 8 Memo
"Deep Learning from scratch" self-study memo (No. 19-2) Data Augmentation continued
Deep Learning / Deep Learning from Zero Chapter 5 Memo
An amateur stumbled in Deep Learning from scratch Note: Chapter 4
Deep Learning / Deep Learning from Zero Chapter 4 Memo
Deep Learning / Deep Learning from Zero 2 Chapter 3 Memo
Deep Learning
An amateur stumbled in Deep Learning from scratch Note: Chapter 2
I tried to implement Perceptron Part 1 [Deep Learning from scratch]
Deep Learning / Deep Learning from Zero 2 Chapter 6 Memo
"Deep Learning from scratch" self-study memo (No. 15) TensorFlow beginner tutorial
Making from scratch Deep Learning ❷ An amateur stumbled Note: Chapter 6
Deep learning tutorial from environment construction
[Deep Learning from scratch] Main parameter update methods for neural networks
Lua version Deep Learning from scratch Part 6 [Neural network inference processing]
Why ModuleNotFoundError: No module named'dataset.mnist' appears in "Deep Learning from scratch".