Understand the tensor product (numpy.tensordot)

I didn't understand numpy's tensor dot catastrophically, but I think I could understand it by researching various things, so I'll make a note of it.

This subject

>>> import numpy as np
>>> a = np.arange(12).reshape(2,3,2)
>>> b = np.arange(48).reshape(3,2,8)
>>> c = np.tensordot(a,b, axes=([1,0], [0,1]))

>>> a
array([[[ 0,  1],
        [ 2,  3],
        [ 4,  5]],

       [[ 6,  7],
        [ 8,  9],
        [10, 11]]])

>>> b
array([[[ 0,  1,  2,  3,  4,  5,  6,  7],
        [ 8,  9, 10, 11, 12, 13, 14, 15]],

       [[16, 17, 18, 19, 20, 21, 22, 23],
        [24, 25, 26, 27, 28, 29, 30, 31]],

       [[32, 33, 34, 35, 36, 37, 38, 39],
        [40, 41, 42, 43, 44, 45, 46, 47]]])

>>> c
array([[ 800,  830,  860,  890,  920,  950,  980, 1010],
       [ 920,  956,  992, 1028, 1064, 1100, 1136, 1172]])

What we are doing with np.tensordot 1: Generate an intermediate matrix from matrix a

1.jpg

The image is that the elements are picked up in the order of the second dimension and the first dimension of the matrix a, and an intermediate matrix for calculation is created. The shape of the intermediate matrix is the number of the second dimension x the number of the first dimension.

68747470733a2f2f71696974612d696d6167652d73746f72652e73332e61702d6e6f727468656173742d312e616d617a6f6e6177732e636f6d2f302f3435353431382f64393430643066642d636365652d373531322d623936652d6638383065613963376462662e6a706567.png

In this case, it is an image that an intermediate matrix with a shape of 3 (second dimension number) x 2 (first dimension number) is created.

3.jpg

4.jpg

In any case, pick up the top left element first.

Which element to pick up next? Since it is picked up in the order of the second dimension and the first dimension, First, pick up in the order of the second dimension.

5.jpg

It will be like this when picked up in the order of the second dimension.

Furthermore, which element to pick up next is I picked it up in the order of the second dimension, so Next, jump to the "lower" island in the order of the first dimension.

6.jpg

As before, pick up the elements in the second dimensional order on the "lower" island.

You now have a 3x2 matrix.

7.jpg

Do the same for the "right" side.

You now have a second 3x2 matrix.

As a result, the following structure is created.

8.jpg

What we are doing with np.tensordot 2: Generate an intermediate matrix from matrix b

9.jpg

The image is that the elements are picked up in the order of the first dimension and the second dimension of the matrix b, and an intermediate matrix for calculation is created.

The shape of the intermediate matrix is the number of the first dimension x the number of the second dimension.

68747470733a2f2f71696974612d696d6167652d73746f72652e73332e61702d6e6f727468656173742d312e616d617a6f6e6177732e636f6d2f302f3435353431382f62613436636630632d326366352d323535362d313130312d3830383662343962366331612e6a706567.png

In this case, it is an image that an intermediate matrix with a shape of 3 (first dimension number) x 2 (second dimension number) is created.

11.jpg

12.jpg

In any case, pick up the top left element first.

Which element to pick up next? Since it is picked up in the order of the first dimension and the second dimension, Next, you have to jump to the "bottom" island.

13.jpg

It will be like this when picked up in the order of the first dimension.

Furthermore, which element to pick up next is I picked it up in the order of the first dimension, so Next, we will return to the "top" island in the order of the second dimension.

14.jpg

While restarting from ④, pick up the elements in the order of the first dimension as before.

You now have a 3x2 matrix.

Do the same for the rest.

You should have eight 3x2 matrices.

As a result, the following structure is created.

15.jpg

What we are doing with np.tensordot 3: Take the "product" of the two intermediate matrices obtained from the matrix a and the matrix b.

16.jpg

17.jpg

18.jpg

It has been made compact into a 2D x 8D matrix.

What are you doing conceptually

68747470733a2f2f71696974612d696d6167652d73746f72652e73332e61702d6e6f727468656173742d312e616d617a6f6e6177732e636f6d2f302f3435353431382f31656164613036652d663366612d393762392d363333392d3166666134353338336339342e6a706567.png

By calculating for these dimensions, we can reduce these dimensions while preserving information on these dimensions.

68747470733a2f2f71696974612d696d6167652d73746f72652e73332e61702d6e6f727468656173742d312e616d617a6f6e6177732e636f6d2f302f3435353431382f66363639666637352d303037632d316534322d396333322d3035653630386663636638302e6a706567.png

It is compacted to the shape of these remaining dimensions (that is, 2 dimensions x 8 dimensions).

In this sense, it seems that such an operation is called contraction. I interpret it as "reduction + summary".

The tensor product seems to be like this.

Recommended Posts

Understand the tensor product (numpy.tensordot)
Understanding the Tensor (2): Shape
Understand the Linux audit system Audit
Understanding the Tensor (3): Real World Data
Understand the contents of sklearn's pipeline
Why super-intelligents couldn't understand the class