[Mathematics] If you understand the meaning of "inner product" graphically, you can see various things.

When you start studying linear algebra, it comes out quite quickly, inner product. The calculation itself is not so difficult, but I think there are people who don't get it right. Me too, I was like that. So, in this visualization series, I will focus on the "inner product".

In addition, there are things that can be understood by the inner product everywhere in statistics, and it becomes "inner product": grin :. A vector is a sequence of data, so it is very related to statistics. It will be treated in statistics from the next time, but first from the inner product.

0. A little conclusion first

Since the previous theory is necessary, I would like to write only the conclusion of the meaning of the inner product first. The inner product can be written as a mathematical formula as follows.

{\bf a}\cdot{\bf b} = \|{\bf a}\|\|{\bf b}\|\ \cos\theta

To express it visually, it becomes as follows, and it can be given the meaning of the length of the vector $ {\ bf a} $ multiplied by the length of the vector $ {\ bf c} . I will. ( {\ bf c} $ is a projection of the vector $ {\ bf b} $ over the vector $ {\ bf a} $.) dotproduct07.png

So far, if you say "Yeah, that's right", please wait for a while as I will explain how the inner product is handled well in statistics next time. (Since the slide version has already been explained, please see ** here ** until then)

If you want to know "why you can say this" and "what this is", I would like to explain it from now on, so please proceed as it is.

1. Definition and calculation of inner product

Let's start with the mathematical definition. When there is a vector $ {\ bf a} = (a_1, \ cdots, a_n), {\ bf b} = (b_1, \ cdots, b_n) $, its inner product is defined as follows.


{\bf a} \cdot {\bf b} = a_1b_1+\cdots+a_nb_n = \sum_{i=1}^n a_ib_i

It means multiplying the corresponding elements of each vector element (the first element $ a_1 $ and the first element $ b_1 $ correspond etc ...) and adding them all together. Let's consider the visual meaning of this.

2. Application of the law of cosines

To understand the inner product, we use what is called the cosine theorem. The formula is below.

\|{\bf b}-{\bf a}\|^2 = \|{\bf a}\|^2 + \|{\bf b}\|^2 - 2\|{\bf a}\|\|{\bf b}\|\ \cos\theta    \cdots (*)

dotproduct02-compressor.png

The relationship between the lengths of each side of the triangle as described above satisfies this formula relationship.

Also, the square of the length of $ {\ bf a} $

\|{\bf a}\|^2 = \left(\sqrt{a_1\cdot a_1 + \cdots + a_n\cdot a_n}\right)^2 =a_1\cdot a_1 + \cdots + a_n\cdot a_n\\
=\sum_{i=1}^n x_i^2 = {\bf a} \cdot {\bf a} 

Can be represented by the dot product of the same vector. this\||{\bf b}-{\bf a}\||^2When applied to

\|{\bf b}-{\bf a}\|^2 = ({\bf b}-{\bf a})\cdot({\bf b}-{\bf a}) = {\bf a}\cdot{\bf a} +  {\bf b}\cdot{\bf b} - 2 {\bf a}\cdot{\bf b} \\
= \|{\bf a}\|^2 + \|{\bf b}\|^2 - 2 {\bf a}\cdot{\bf b}         

Substituting this into the (*) equation of the cosine theorem

\|{\bf b}-{\bf a}\|^2 = \|{\bf a}\|^2 + \|{\bf b}\|^2 - 2\|{\bf a}\|\|{\bf b}\|\ cos\theta\\
\Rightarrow \|{\bf a}\|^2 + \|{\bf b}\|^2 - 2 {\bf a}\cdot{\bf b} = \|{\bf a}\|^2 + \|{\bf b}\|^2 - 2\|{\bf a}\|\|{\bf b}\|\ cos\theta\\

On both sides\||\{\bf a}\||^2 + \||\{\bf b}\||\^2Because there is, erase it,

 - 2 {\bf a}\cdot{\bf b} = - 2\|{\bf a}\|\|{\bf b}\|\ cos\theta \\
\Rightarrow {\bf a}\cdot{\bf b} = \|{\bf a}\|\|{\bf b}\|\ cos\theta

So, the inner product is

\|{\bf a}\|\|{\bf b}\|\ \cos \theta

It turned out to be.
Wow, I see ...? ?? ??
So what? I'm feeling··· But this shape is very important, so please remember it. Then this\||{\bf a}\||\||{\bf b}\||\ \cos\thetaLet's see who is.

3.||a|| ||b||Explore the mystery of cosθ#

3.1 Reconfirmation of cos θ and the concept of projection

First of all, I would like to reconfirm $ \ cos \ theta $ once again. The mathematical definition is

\cos \theta = \frac{\|c\|}{\|r\|}

is. As shown in the graph, dotproduct11-compressor.png

$ r $ is the radius and $ x $ is the vector from the vector $ r $ to that point, drawing a line perpendicular to the x-axis. As an image, as shown below, it is the part that illuminates the light from directly above and becomes a shadow. (It's called projection.) ** The shadow part created by illuminating the roof $ {\ bf r} $ is the vector $ {\ bf c} $. ** **

dotproduct10-compressor.png

3.2 Meaning of inner product

\cos \theta = \||c\||\ /\ ||r\||On both sides of\||r\||And the denominator\||r\||To the opposite section,

\|c\| = \|r\|\cos \theta

So the length of c is\||r\||\cos \thetaCan be considered to be.To the length of the roof\cos \thetaIt is thought that it was multiplied by.

By the way, here was the homework of the previous section

\|{\bf a}\|\|{\bf b}\|\ \cos \theta

Let's get back to. Similarly, try shining a light from above.

dotproduct07.png

vector{\bf b}からvector{\bf a}上に射影してできたvectorを{\bf c}Then, if you apply the logic above, the length will be "to the length of the roof.\cos \thetaBecause it is a product of\||{\bf b}\|| \cos \thetaWill be. Now you have all the ingredients you need. Inner product $ {\ bf a} \ cdot {\ bf b} $

{\bf a}\cdot{\bf b} = \|{\bf a}\|\|{\bf b}\|\ \cos\theta

So, ** "length of vector $ {\ bf a} $" ** and ** "length of vector $ {\ bf b} $ projected onto vector $ {\ bf a} $" It is considered to be the product of **.

Since the inner product is also ** "product" **, it is some kind of multiplication, but I think that the image of ** multiplication between vectors should be the image of matching the directions and multiplying the length **. I am.

3.3 When the directions of the two vectors are vertical

dotproduct08-compressor.png

If the angles of the two vectors are right angles, you can't cast a shadow even if you shine a light from above. Therefore, at this time, the length of the vector $ {\ bf c} $ becomes 0. In other words, when the angle is right angle, the inner product is also 0.

3.3 For a vector of length 1

Make the story easier||r||=1, ||x||=1Suppose it was.\cos \thetaThe definition of

\cos \theta = \frac{\|c\|}{\|r\|} = \|c\|

Therefore, the length of $ {\ bf c} $ becomes the value of $ \ cos \ theta $ as it is.

dotproduct12-compressor.png Now, if you look at the inner product of $ {\ bf x} $ and $ {\ bf r} $,

{\bf x}\cdot{\bf r} = \|{\bf x}\|\|{\bf r}\|\ \cos\theta = 1 \cdot 1 \cdot \cos\theta = \cos\theta 

Therefore, the inner product of two vectors with lengths of 1 is $ \ cos \ theta $ **: smile:

In the next article, I will write about "scenes where inner products are used in statistics". We will explain how the variance / standard deviation, covariance / correlation coefficient, regression coefficient, etc. look in relation to this inner product using graphs and animations in Python.

Recommended Posts

[Mathematics] If you understand the meaning of "inner product" graphically, you can see various things.
You can see the transition of points in the J League on the graph!
You can see it! Comparison of optimization methods (2020)
python> "I don't understand the meaning of", = "> (Solution) One Element Tuples: You answered the comment