When I was studying mathematics, the word "inner product" came up. I investigated how this "inner product" is used in neural networks. The inner product is the product of vectors, but the matrix product is the product of matrices. The inner product seems to be called "dot product".
-Meaning of inner product -[Mathematics] If you understand the meaning of "inner product" graphically, you can see various things. 1 -[Dot product](https://ja.wikipedia.org/wiki/dot product)
-Definition of matrix product and its reason
Consider the following two-layer neural network.
If you replace the output of the above Nural network with the matrix product formula, it looks like this.
\begin{bmatrix}
w_{11} & w_{12} & w_{13}\\
w_{21} & w_{22} & w_{23}
\end{bmatrix}
\cdot
\begin{bmatrix}
x_{1} \\
x_{2} \\
x_{3} \\
\end{bmatrix}
Apply the numbers specifically
\begin{bmatrix}
1 & 2 & 3\\
4 & 5 & 6
\end{bmatrix}
\cdot
\begin{bmatrix}
1 \\
2 \\
3 \\
\end{bmatrix}
If you calculate by hand
\begin{bmatrix}
1 \times 1 + 2 \times 2 + 3 \times 3 \\
4 \times 1 + 5 \times 2 + 6 \times 3 \\
\end{bmatrix}
=
\begin{bmatrix}
14 \\
32 \\
\end{bmatrix}
import numpy as np
w = np.array([[1,2,3],[4,5,6]])
x = np.array([[1],[2],[3]])
w.dot(x) #=> array([[14],[32]])
w.dot (x)
represents the matrix product of w and x.
By the way, the method name is changed from "dot product" to dot
.
import tensorflow as tf
w = tf.get_variable("weight" , initializer=tf.constant(np.array([[1,2,3],[4,5,6]]).astype("int32")))
x = tf.get_variable("x" , initializer=tf.constant(np.array([[1],[2],[3]]).astype("int32")))
Session initialization
init = tf.initialize_all_variables()
sess = tf.Session()
sess.run(init)
sess.run(tf.matmul(w, x)) #=> array([[14],[32]])
tf.matmul (w, x)
represents the matrix product of w and x.
By the way, the method name is matmul
from" matrix multiplication ".
Recommended Posts