Duality in function

Vectors and co-vectors are equal and their roles can be swapped. This is called duality. Let's check with an example. Attach the calculation by NumPy.

This is a series of articles.

1. Inner product considered by estimation
2. Covector to think with functions
3. Matrix to think with functions
4. Duality in function ← This article
5. Perceptron Thinking in Covector

It is assumed that NumPy is imported as follows.

>>> from numpy import *

Duality

Previously, I considered the following spreadsheet.

To calculate this, prepare a matrix and a vector.

>>> A=array([[30,50,150],[25,60,120]]).T
>>> A
array([[ 30,  25],
       [ 50,  60],
       [150, 120]])
>>> X=array([[12,10,5]]).T
>>> X
array([[12],
       [10],
       [ 5]])

A=\left(\begin{matrix} 30 & 25 \\ 50 & 60 \\ 150 & 120 \end{matrix}\right),
X=\left(\begin{matrix} 12 \\ 10 \\ 5 \end{matrix}\right)

Notice that it has the same shape as the spreadsheet. If you transpose the left side, you can get the same number even if you change the order.

>>> dot(A.T,X)
array([[1610],
       [1500]])
>>> dot(X.T,A)
array([[1610, 1500]])

A^{\top}X
=\left(\begin{matrix} 30 & 50 & 150 \\ 25 & 60 & 120 \end{matrix}\right)
 \left(\begin{matrix} 12 \\ 10 \\ 5 \end{matrix}\right)
=\left(\begin{matrix} 1610 \\ 1500 \end{matrix}\right) \\
X^{\top}A
=\left(\begin{matrix} 12 & 10 & 5 \end{matrix}\right)
 \left(\begin{matrix} 30 & 25 \\ 50 & 60 \\ 150 & 120 \end{matrix}\right)
=\left(\begin{matrix} 1610 & 1500 \end{matrix}\right) \\

This is interpreted as swapping the roles of vector and covector. If you pass a vector as an argument, a vector will be returned, and if you pass a covector, a covector will be returned.

Formally, when the whole is transposed, the order is reversed and they are transposed individually.

(A^{\top}X)^{\top}=X^{\top}A

It is similar to reversing the sign of the whole by subtraction, but the order is not reversed if the sign is reversed individually.

\begin{align*}
-(3-2)
&=2-3 \\
&=(-3)-(-2)
\end{align*}

Number of inputs / outputs

The calculation flow is reversed for vectors and co-vectors.

\underbrace{GF}_{Synthetic}
=\overbrace{\left(\begin{matrix}g_1 & g_2 & g_3\end{matrix}\right)}^{Number of inputs 3}
 \quad\scriptsize{Number of outputs 3}\normalsize{\Biggr\{
  \left(\begin{matrix}f_{11} & f_{12} \\ f_{21} & f_{22} \\ f_{31} & f_{32}\end{matrix}\right)} \\
\overbrace{\left(\begin{matrix}f_{11} & f_{12} & f_{13} \\ f_{21} & f_{22} & f_{23} \end{matrix}\right)}^{Number of outputs 3}
 \quad\scriptsize{Number of inputs 3}\normalsize{\Biggr\{
  \left(\begin{matrix}g_1 \\ g_2 \\ g_3 \end{matrix}\right)}
=\underbrace{FG}_{Synthetic}

If you remove the number of elements in between, only the inputs and outputs remain.

\underbrace{1}_{output}×\underbrace{3←3}_{Removal}×\underbrace{2}_{input} \\
\underbrace{2}_{input}×\underbrace{3→3}_{Removal}×\underbrace{1}_{output}

Calculation flow

The flow of calculation including input and output is shown. The red number represents the number of elements.

Schematize by focusing on the value.

\underbrace{y}_{output}
\xleftarrow{G}
\underbrace{\left(\begin{matrix}t_1 \\ t_2 \\ t_3\end{matrix}\right)}_{Value in the middle}
\xleftarrow{F}
\underbrace{\left(\begin{matrix}x_1 \\ x_2\end{matrix}\right)}_{input} \\
\underbrace{\left(\begin{matrix}x_1 & x_2\end{matrix}\right)}_{input}
\xrightarrow{F}
\underbrace{\left(\begin{matrix}t_1 & t_2 & t_3\end{matrix}\right)}_{Value in the middle}
\xrightarrow{G}
\underbrace{y}_{output}

Draw a figure in the style of Perceptron. The calculation flow is from right to left according to the matrix notation.

Notice the connection between $ t $ and $ x $. Taking $ f_ {21} $ as an example, we can see that by interpreting the subscript as $ 2 ← 1 $, it corresponds to the subscript of the node $ t_2 ← x_1 $ connected by the line. The schematic is as follows.

t_2 \xleftarrow{f_{21}} x_1 \\
x_1 \xrightarrow{f_{12}} t_2 \\

Make sure that the other lines have the same pattern.