Covector to think in function

Explain the concept that a function can be implemented using the dot product of vectors. [Algebraic Dual Space](https://en.wikipedia.org/wiki/%E5%8F%8C%E5%AF%BE%E3%83%99%E3%82%AF%E3%83%88%E3 The purpose is to convey the image of% 83% AB% E7% A9% BA% E9% 96% 93). Python is used as the programming language for comparison, and the result of NumPy is attached to the calculation of the inner product.

This is a series of articles.

  1. Inner product considered by estimation
  2. Thinking with a function co-vector ← This article
  3. Matrix considered by function
  4. Duality in function
  5. Perceptron Thinking in Covector

This article has related articles.

inner product

The inner product of $ \ vec {a} \ cdot \ vec {b} $ is regarded as the action of $ \ vec {a} \ cdot $ from the left on $ \ vec {b} $. It can be expressed as a product by transposing $ \ vec {a} $.

\begin{align}
\underbrace{\vec{a}\cdot}_{Action}\vec{b}
&=\underbrace{\left(\begin{matrix}a_1 \\ a_2 \\ a_3\end{matrix}\right)\cdot}_{Action}
  \left(\begin{matrix}b_1 \\ b_2 \\ b_3\end{matrix}\right) \\
&=\underbrace{\left(\begin{matrix}a_1 \\ a_2 \\ a_3\end{matrix}\right)^{\top}}_{Transpose}
  \left(\begin{matrix}b_1 \\ b_2 \\ b_3\end{matrix}\right) \\
&=\underbrace{\left(\begin{matrix}a_1 &  a_2 &  a_3\end{matrix}\right)}_{Horizontal vector}
  \underbrace{\left(\begin{matrix}b_1 \\ b_2 \\ b_3\end{matrix}\right)}_{Vertical vector} \\
&=\underbrace{a_1b_1+a_2b_2+a_3b_3}_{inner product}
\end{align}

It is good to be aware that "horizontal vector times vertical vector" is another expression of the inner product of vectors. This pattern also derives from the fact that the direction of matrix multiplication is "horizontal → vertical".

Covector

The horizontal vector that hangs on the vertical vector from the left is called covector. I will. The vertical vector to the right of the covector is simply called the vector. Based on the vertical vector, the transposed horizontal vector is prefixed with "co-" to represent duality. (Same as ** co ** for signature ** co ** for signature)

\underbrace{\left(\begin{matrix}a_1 &  a_2 &  a_3\end{matrix}\right)}_{Covector}
\underbrace{\left(\begin{matrix}b_1 \\ b_2 \\ b_3\end{matrix}\right)}_{vector}
\underbrace{\left(\begin{matrix}a_1 &  a_2 &  a_3\end{matrix}\right)}_{1-format}
\underbrace{\left(\begin{matrix}b_1 \\ b_2 \\ b_3\end{matrix}\right)}_{1-vector}

The covector (1-form) has a special meaning. Simply put, transpose has the function of converting a vector (matrix) into a function.

\underbrace{\left(\begin{matrix}a_1 &  a_2 &  a_3\end{matrix}\right)}_{Function (implementation)}
\underbrace{\left(\begin{matrix}b_1 \\ b_2 \\ b_3\end{matrix}\right)}_{argument}
=\underbrace{a_1b_1+a_2b_2+a_3b_3}_{Return value}

function

Even though a covector can be considered a function, the functions that can be implemented are limited to those that can be represented by an inner product. Here are some examples.

getY

Returns the second of the three arguments.

>>> def getY(x, y, z):
...     return y
...
>>> getY(1,2,3)
2

Implement with covector. Start with a capital letter to distinguish it.

>>> from numpy import *
>>> GetY = [0, 1, 0]
>>> dot(GetY, [1, 2, 3])
2
GetY
 \left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
=\left(\begin{matrix}0 &  1 &  0\end{matrix}\right)
 \left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
=2

The inner product with the covector gave the same result.

You can implement GetX and GetZ as well.

>>> GetX = [1, 0, 0]
>>> GetZ = [0, 0, 1]
>>> dot(GetX, [1, 2, 3])
1
>>> dot(GetZ, [1, 2, 3])
3

sum

Returns the sum of the three arguments.

>>> def sum(x, y, z):
...     return x + y + z
...
>>> sum(1, 2, 3)
6

Implement with covector.

>>> Sum = [1, 1, 1]
>>> dot(Sum, [1, 2, 3])
6
Sum
 \left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
=\left(\begin{matrix}1 &  1 &  1\end{matrix}\right)
 \left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
=6

average

Returns the mean of the three arguments.

>>> def average(x, y, z):
...     return (x + y + z) / 3
...
>>> average(1, 2, 3)
2.0

Implement with covector.

>>> Average = array([1, 1, 1]) / 3
>>> dot(Average, [1, 2, 3])
2.0
Average
 \left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
=\frac{1}{3}
 \left(\begin{matrix}1 &  1 &  1\end{matrix}\right)
 \left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
=2

gray

Converts RGB values to grayscale.

>>> def gray(r, g, b):
...     return r * 0.299 + g * 0.587 + b * 0.114
...
>>> gray(64, 128, 192)
116.16

Implement with covector.

>>> Gray = [0.299, 0.587, 0.114]
>>> dot(Gray, [64, 128, 192])
116.16
Gray
 \left(\begin{matrix}64 \\ 128 \\ 192\end{matrix}\right)
=\left(\begin{matrix}0.299 & 0.587 & 0.114\end{matrix}\right)
 \left(\begin{matrix}64 \\ 128 \\ 192\end{matrix}\right)
=116.16

Specifying the ratio and adding them together is the original usage of the inner product.

Bad example

The inner product can only be expressed by linear operations (constant times and sum).

There are many things you can't do, but for example, the following functions cannot be implemented with covectors.

>>> def product(x, y, z):
...     return x * y * z
...
>>> product(2, 3, 4)
24

Multiple calculations

Although it is a covector function with limited expressiveness, it has the advantage of being able to combine multiple calculations. Let's confirm that.

Check by classifying by the ratio of "function: argument".

One-to-many

It's common to pass different arguments to a function for calculations.

>>> sum(1,2,3)
6
>>> sum(4,5,6)
15

List comprehensions can be used to combine multiple such calculations into one.

>>> [sum(x,y,z) for x,y,z in [(1,2,3),(4,5,6)]]
[6, 15]

In the calculation using covector, you can calculate at once by passing a matrix in which the argument vectors are arranged side by side.

>>> dot(Sum,[[1,4],[2,5],[3,6]])
array([ 6, 15])
\begin{align}
Sum\left(\begin{array}{c|c}1 & 4 \\ 2 & 5 \\ 3 & 6\end{array}\right)
&=\left(\begin{matrix}
    Sum\left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right) &
    Sum\left(\begin{matrix}4 \\ 5 \\ 6\end{matrix}\right)
  \end{matrix}\right) \\
&=\left(\begin{matrix}6 & 15\end{matrix}\right)
\end{align}

Only after using this method will the joy of matrix calculation come out. However, the description in NumPy is confusing because the two argument sets are mixed up. It is better to write using transpose.

>>> dot(Sum,array([[1,2,3],[4,5,6]]).T)
array([ 6, 15])

Many to one

Now consider passing the same arguments to multiple functions.

>>> sum(1,2,3)
6
>>> average(1,2,3)
2.0

This can also be written in list comprehension.

>>> [(sum(x,y,z),average(x,y,z)) for x,y,z in [(1,2,3)]]
[(6, 2.0)]

Arguments can be shared by arranging covectors vertically.

>>> dot([Sum,Average],[1,2,3])
array([ 6.,  2.])
\begin{align}
\left(\begin{matrix}Sum \\ Average\end{matrix}\right)
\left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
&=\left(\begin{array}{r}
    Sum    \left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right) \\
    Average\left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right)
  \end{array}\right) \\
&=\left(\begin{matrix}6 \\ 2\end{matrix}\right)
\end{align}

Many-to-many

As you might expect, you can calculate all combinations at once by using multiple function and argument pairs.

Solid writing


>>> sum(1,2,3)
6
>>> average(1,2,3)
2.0
>>> sum(4,5,6)
15
>>> average(4,5,6)
5.0

List comprehension


>>> [(sum(x,y,z),average(x,y,z)) for x,y,z in [(1,2,3),(4,5,6)]]
[(6, 2.0), (15, 5.0)]

Arrange the covector and the argument vertically, and transpose the argument.

>>> dot([Sum,Average],array([[1,2,3],[4,5,6]]).T)
array([[  6.,  15.],
       [  2.,   5.]])
\begin{align}
\left(\begin{matrix}Sum \\ Average\end{matrix}\right)
\left(\begin{array}{c|c}1 & 4 \\ 2 & 5 \\ 3 & 6\end{array}\right)
&=\left(\begin{array}{rr}
    Sum    \left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right) &
    Sum    \left(\begin{matrix}4 \\ 5 \\ 6\end{matrix}\right) \\
    Average\left(\begin{matrix}1 \\ 2 \\ 3\end{matrix}\right) &
    Average\left(\begin{matrix}4 \\ 5 \\ 6\end{matrix}\right)
  \end{array}\right) \\
&=\left(\begin{matrix}6 & 15 \\ 2 & 5\end{matrix}\right)
\end{align}

Covectors and dual spaces may seem like abstract ideas that are unrealistic, but if you mean them in this way, you will get a concrete image.

reference

I used the Wikipedia example with some modifications.

The idea of covectors comes from the dual vector space.

Recommended Posts

Covector to think in function
Duality in function
Automatically register function arguments to argparse in Python
To execute a Python enumerate function in JavaScript
How to Mock a Public function in Pytest
Execute function in parallel
Generator function in JavaScript
Perceptron thinking in covector
Function to return multi columns to single column in DataFrame
How to define Decorator and Decomaker in one function
[Road to Python Intermediate] Define __getattr__ function in class
Create a function in Python
To flush stdout in Python
Use callback function in Python
ntile (decile) function in python
Login to website in Python
Implement timer function in pygame
How to call a function
Think about architecture in python
A handy function to add a column anywhere in a Pandas DataFrame
How to sample from any probability density function in Python
To return char * in a callback function using ctypes in Python
Nonlinear function modeling in Python
Draw implicit function in python
Speech to speech in python [text to speech]
In omegaconf, let's pass the direct parameter file to the function
I tried to implement the mail sending function in Python
Immediate function in python (lie)
Let's create a function to hold down Button in Tkinter
How to develop in Python
Post to Slack in Python
I wrote a function to load a Git extension script in Python
Difference in execution speed depending on how to write Cython function
Think seriously about what language to use in programming education and programming education.
Create a function to get the contents of the database in Go
Precautions when giving default values to arguments in Python function definitions
Attempt to extend a function in the library (add copy function to pathlib)
How to use the render function defined in .mako (.html) directly in mako
Programming to fight in the world ~ 5-1
Programming to fight in the world ~ 5-5,5-6
Programming to fight in the world 5-3
[Python] How to do PCA in Python
How to handle session in SQLAlchemy
How to use the zip function
Function to convert Excel column to number
Implement R's power.prop.test function in python
Create user authentication function in Airflow
sort warning in the pd.concat function
What beginners think about programming in 2016
Function argument type definition in python
Included notation in Python function arguments
Convert markdown to PDF in Python
How to use classes in Theano
How to write soberly in pandas
How to collect images in Python
Mock in python-how to use mox
Errors related to memcached in django
Steps to install Ubuntu in VirtualBox
Set opset to embed in ONNX
Implementation of login function in Django
How to update Spyder in Anaconda