Perceptron thinking in covector

We will review the idea of inner products by spreadsheets and covectors by functions, and look at logical operations by perceptrons from that point of view. Attach the calculation by NumPy.

This is a series of articles.

  1. Inner product considered by estimation
  2. Covector to think with functions
  3. Matrix considered by function
  4. Duality in function
  5. Thinking with covector Perceptron ← This article

This article was written with the intention of assisting the following books:

Estimate calculation

I will repost the previous example.

Product name unit price(Company A) Quantity subtotal
pencil 30 12 360
eraser 50 10 500
Note 150 5 750
Grand total 1,610

If you rewrite the unit price and quantity as a vector, this is the calculation of the inner product.

\overbrace{\left(\begin{matrix}30 \\ 50 \\150\end{matrix}\right)}^{unit price}\cdot
\overbrace{\left(\begin{matrix}12 \\ 10 \\ 5\end{matrix}\right)}^{Quantity}
=\overbrace{30×12}^{subtotal}+\overbrace{50×10}^{subtotal}+\overbrace{150×5}^{subtotal}
=\overbrace{1610}^{Grand total}

NumPy

Here is the calculation in NumPy.

>>> from numpy import *
>>> dot([30,50,150],[12,10,5])
1610

Do not ask for the total at once, but find the subtotal by Hadamard product.

>>> array([30,50,150])*[12,10,5]
array([360, 500, 750])

The total of the subtotals gives the total.

>>> sum(array([30,50,150])*[12,10,5])
1610

Functionalization by covector

By assigning the unit price of company A to a horizontal vector (covector) called ʻA`, taking the inner product can be equated with evaluating a function.

>>> A=[30,50,150]
>>> dot(A,[12,10,5])
1610
A
 \left(\begin{matrix}12 \\ 10 \\ 5\end{matrix}\right)
=\left(\begin{matrix}30 &  50 &  150\end{matrix}\right)
 \left(\begin{matrix}12 \\ 10 \\ 5\end{matrix}\right)
=1610

Comparison of number

Consider a case where the number is changed and compared.

Product name unit price Quantity ① Quantity ② Subtotal ① Subtotal ②
pencil 30 12 9 360 270
eraser 50 10 13 500 650
Note 150 5 4 750 600
Grand total 1,610 1,520

In the calculation using covector, you can calculate at once by passing a matrix in which the argument vectors are arranged side by side.

>>> dot(A,array([[12,10,5],[9,13,4]]).T)
array([1610, 1520])
\begin{align}
A\left(\begin{array}{c|c}12 & 9 \\ 10 & 13 \\ 5 & 4\end{array}\right)
&=\left(\begin{matrix}
    A\left(\begin{matrix}12 \\ 10 \\ 5\end{matrix}\right) &
    A\left(\begin{matrix} 9 \\ 13 \\ 4\end{matrix}\right)
  \end{matrix}\right) \\
&=\left(\begin{matrix}1610 & 1520\end{matrix}\right)
\end{align}

This is the review.

AND

Consider ʻAND` as a logical operation. Shows the Cayley Table.

A B A&B
0 0 0
0 1 0
1 0 0
1 1 1

This is a kind of calculation that cannot be expressed by covector. Therefore, post-processing is added.

The idea is to add A and B, find out if it is greater than 1, and if true it is 1.

A B A+B >1 A&B
0 0 0 F 0
0 1 1 F 0
1 0 1 F 0
1 1 2 T 1

The part where A and B are added is represented by the covector ʻAnd0`, the magnitude comparison is added, and zero is added for quantification.

>>> And0=[1,1]
>>> (dot(And0,array([[0,0],[0,1],[1,0],[1,1]]).T)>1)+0
array([0, 0, 0, 1])

This covector $ (1 \ 1) $ is called ** weight ** $ w $, and > 1 $ 1 $ is called threshold $ θ $.

perceptron

The calculation based on weights and thresholds is called a perceptron. Defines a perceptron function.

>>> def perceptron(w, th, x):
...     return (dot(w, x) > th) + 0
...

Use this to calculate ʻAND`.

>>> perceptron(And0, 1, [0,1])
0

Like the covector, it accepts multiple arguments.

>>> perceptron(And0, 1, array([[0,0],[0,1],[1,0],[1,1]]).T)
array([0, 0, 0, 1])

Recommended Posts

Perceptron thinking in covector
Covector to think in function
Implemented Perceptron learning rules in Python
perceptron