Matrix considered by function

Last time, I saw that covectors can be used to combine multiple functions that return scalars. This time, the function that returns a vector is represented by a matrix. [Algebraic Dual Space](https://en.wikipedia.org/wiki/%E5%8F%8C%E5%AF%BE%E3%83%99%E3%82%AF%E3%83%88%E3 The purpose is to convey the image of% 83% AB% E7% A9% BA% E9% 96% 93). Use Python as the programming language for comparison and attach the NumPy results to the calculations.

This is a series of articles.

  1. Inner product considered by estimation
  2. Covector to think with functions
  3. Matrix considered by function ← This article
  4. Duality in function
  5. Perceptron Thinking in Covector

Covector function

I will repost the covector that returns the value specified from the three arguments introduced last time.

>>> from numpy import *
>>> GetX = [1, 0, 0]
>>> GetY = [0, 1, 0]
>>> GetZ = [0, 0, 1]
>>> dot(GetX, [1, 2, 3])
1
>>> dot(GetY, [1, 2, 3])
2
>>> dot(GetZ, [1, 2, 3])
3

\begin{align}
GetX
&\left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
=\left(\begin{matrix}1 &  0 &  0\end{matrix}\right)
 \left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
=1 \\
GetY
&\left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
=\left(\begin{matrix}0 &  1 &  0\end{matrix}\right)
 \left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
=2 \\
GetZ
&\left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
=\left(\begin{matrix}0 &  0 &  1\end{matrix}\right)
 \left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
=3
\end{align}

Many to one

Last time I saw that you can pass the same arguments to multiple functions. I will try it with the previous function.

>>> dot([GetX,GetY,GetZ],[1,2,3])
array([1, 2, 3])

\begin{align}
\left(\begin{matrix}GetX \\ GetY \\ GetZ\end{matrix}\right)
\left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
&=\left(\begin{array}{r}
    GetX \left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right) \\
    GetY \left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right) \\
    GetZ \left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
  \end{array}\right) \\
&=\left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
\end{align}

If you consider the entire return value as a vector, you will notice that it is returning the arguments as they are. It is a natural result to note that the matrix composed of covectors becomes the identity matrix.

\left(\begin{matrix}GetX \\ GetY \\ GetZ\end{matrix}\right)
=\left(\begin{matrix}1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1\end{matrix}\right)

Matrix function

If you think of the previous example as one result of one function, you can think of the matrix composed of covectors as one function.

>>> Id=[GetX,GetY,GetZ]
>>> dot(Id,[1,2,3])
array([1, 2, 3])

Id
 \left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
=\left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)

You can also create a function to cycle by shifting the arrangement.

>>> Rotate=[GetY,GetZ,GetX]
>>> dot(Rotate,[1,2,3])
array([2, 3, 1])

Rotate
 \left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
=\left(\begin{matrix}2 \\ 3 \\ 1\end{matrix}\right)

In this way, you can think of a matrix as a function and interpret it by dividing it into covectors as needed.

Bondability

It is also possible to shift it twice in a row.

>>> dot(Rotate,dot(Rotate,[1,2,3]))
array([3, 1, 2])

Rotate\left(Rotate
 \left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)\right)
=\left(\begin{matrix}3 \\ 1 \\ 2\end{matrix}\right)

It is possible to calculate the product of Rotates first, just as the result is the same even if the combination to be calculated first is changed such as $ a (bc) = (ab) c $ by multiplication.

>>> dot(dot(Rotate,Rotate),[1,2,3])
array([3, 1, 2])

\left\{(Rotate)(Rotate)\right\}
 \left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
=\left(\begin{matrix}3 \\ 1 \\ 2\end{matrix}\right)

The fact that the result does not change even if the calculation order is changed in this way is called ** connectivity **. The method of calculating the product of matrices is designed to ensure associativity.

Function synthesis

It is possible to consider the product of functions by a matrix as ** function composition ** and assign it to a new function.

>>> Rotate2=dot(Rotate,Rotate)
>>> dot(Rotate2,[1,2,3])
array([3, 1, 2])

Rotate2
 \left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
=\left(\begin{matrix}3 \\ 1 \\ 2\end{matrix}\right)

If you Rotate three times, it will return to the original, but you can see that the result of function synthesis is an identity matrix.

>>> Rotate3=dot(Rotate2,Rotate)
>>> dot(Rotate3,[1,2,3])
array([1, 2, 3])
>>> Rotate3
array([[1, 0, 0],
       [0, 1, 0],
       [0, 0, 1]])

(Rotate2)(Rotate)
=\left(\begin{matrix}1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1\end{matrix}\right)

Number of inputs / outputs

If you think of a matrix as a function, you can interpret that the size of the matrix represents the number of elements in the argument (number of inputs) and the number of elements in the return value (number of outputs).

Input → Function → Output


(X, X)    -> F -> (X, X, X)
(X, X, X) -> G ->  X

\begin{align}
F&:\quad\scriptsize{Number of outputs 3}\normalsize{\Biggr\{
  \overbrace{\left(\begin{matrix}f_{11} & f_{12} \\ f_{21} & f_{22} \\ f_{31} & f_{32}\end{matrix}\right)}^{Number of inputs 2}} \\
G&:\quad\scriptsize{Number of outputs 1}\normalsize{\{
  \overbrace{\left(\begin{matrix}g_1 & g_2 & g_3\end{matrix}\right)}^{Number of inputs 3}}
\end{align}

If you want to calculate $ F $ and $ G $ in succession, the number of outputs of $ F $ calculated first must match the number of inputs of subsequent $ G $.

F→
\underbrace{\left(\begin{matrix}t_1 \\ t_2 \\ t_3\end{matrix}\right)}_{Value in the middle}
→G

Check the component display to see the situation where $ F $ and $ G $ are combined into $ GF $.

\underbrace{GF}_{Synthetic}
=\overbrace{\left(\begin{matrix}g_1 & g_2 & g_3\end{matrix}\right)}^{Number of inputs 3}
 \quad\scriptsize{Number of outputs 3}\normalsize{\Biggr\{
  \left(\begin{matrix}f_{11} & f_{12} \\ f_{21} & f_{22} \\ f_{31} & f_{32}\end{matrix}\right)}

Calculation flow

The flow of calculation including input and output is shown. The red number represents the number of elements.

図1.png

Schematize by focusing on the value.

\underbrace{y}_{output}
\xleftarrow{G}
\underbrace{\left(\begin{matrix}t_1 \\ t_2 \\ t_3\end{matrix}\right)}_{Value in the middle}
\xleftarrow{F}
\underbrace{\left(\begin{matrix}x_1 \\ x_2\end{matrix}\right)}_{input}

Draw a figure in the style of Perceptron. The calculation flow is from right to left according to the matrix notation.

図2.png

Notice the line connecting $ t $ and $ x $. Taking $ f_ {21} $ as an example, we can see that by interpreting the subscript as $ 2 ← 1 $, it corresponds to the subscript of the node $ t_2 ← x_1 $ connected by the line. The schematic is as follows.

t_2 \xleftarrow{f_{21}} x_1

Make sure that the other lines have the same pattern.

Tensor flow

If you use the node subscript as a variable and write it as $ x_i, t_j $, the line connecting them will be $ f_ {ji} $.

t_j \xleftarrow{f_{ji}} x_i

If you include the last step, the flow will be as follows.

y \xleftarrow{g_j} t_j \xleftarrow{f_{ji}} x_i

** Tensor algebra ** expresses this relationship with the following formula.

t_j=f_{ji}x_i \\
y=g_jt_j=g_jf_{ji}x_i

The representation of tensor algebra is a matrix with subscript relationships added.

\boldsymbol{t}=\boldsymbol{Fx} \\
y=\boldsymbol{Gt}=\boldsymbol{GFx}

If you express the node value in tensor algebra, you can see the flow of the tensor.

\underbrace{g_jf_{ji}x_i}_{output}
\xleftarrow{g_j}
\underbrace{f_{ji}x_i}_{Value in the middle}
\xleftarrow{f_{ji}}
\underbrace{x_i}_{input}

図3.png

Recommended Posts

Matrix considered by function
Quaternion considered in representation matrix
Function to save images by date [python3]
ECG data anomaly detection by Matrix Profile