[Pytorch] Utilization of einsum to ease the product of matrices (tensors)

If you read the article and have any suggestions, please feel free to contact us. It will be encouraging if you like it.

What is einsum

--ʻEinsumis theEinstein notationimplemented innumpy and pytorch`. (I didn't know how to call it like this in Japanese.) ――It makes it possible to operate complicated tensor product operations very intentionally.

background

--Until I used ʻeinsum, I used torch.bmmandtorch.matmul (only the ones I just came up with) as the method of calculating the product of tensors in pytorch`. ――However, the dimension of the tensor is predetermined in the product calculation method mentioned above. (For example, 3D tensor x 2D tensor, 2D tensor x 1D tensor, etc.) ――In this case, it was difficult to check the document one by one and use the function (method) according to the dimension.

Commentary

pattern 1

Arithmetic that tends to be deep learning

import torch as t
X = t.rand(3,10,5)
Y = t.rand(3,20,5)

--Mini batch is $ 3 $ --The size of matrix X is $ 10 \ times5 $ --The size of matrix Y is $ 20 \ times5 $

In such a case, I want to calculate the product of each matrix in the mini-batch and calculate the matrix with the size of $ 10 \ times20 $. And you may want to return it as a batch.

--In other words, I want you to return the tensor $ 3 \ times10 \ times20 $ as the result of the XY operation. --As a result, you can calculate as follows

t.einsum('bnm,bkm->bnk',X,Y).size()                      
>> torch.Size([3, 10, 20])

--By the way, when you want to return $ 3 \ times20 \ times10 $ instead of $ 3 \ times10 \ times20 $ --'bnm, bkm-> bkn' Pay attention to the difference here

t.einsum('bnm,bkm->bkn',X,Y).size()                      
>> torch.Size([3, 20, 10])

Pattern 2

You can also calculate the product of a matrix and a vector. simply.

X = t.rand(3,10,5)                                       
Y = t.rand(3,5)                                          
t.einsum('bnm,bm->bn',X,Y).size()                        
>> torch.Size([3, 10])

Summary

――By using ʻeinsum, you can express the product of complicated tensors very easily. This is goodbye to those who forcibly matched the dimensions with torch.transpose, torch.view, torch.squeeze`, etc.

Recommended Posts

[Pytorch] Utilization of einsum to ease the product of matrices (tensors)
Calculate the product of matrices with a character expression?
About the ease of Python
I want to manually assign the training parameters of the [Pytorch] model
Supplement to the explanation of vscode
The story of trying to reconnect the client
Script to change the description of fasta
10 methods to improve the accuracy of BERT
How to check the version of Django
The story of adding MeCab to ubuntu 16.04
Basics of PyTorch (1) -How to use Tensor-
I want to use PyTorch to generate something like the lyrics of Japari Park
Change the data frame of pandas purchase data (id x product) to a dictionary