If you read the article and have any suggestions, please feel free to contact us. It will be encouraging if you like it.
--ʻEinsumis theEinstein notationimplemented innumpy and pytorch`. (I didn't know how to call it like this in Japanese.)
――It makes it possible to operate complicated tensor product operations very intentionally.
--Until I used ʻeinsum, I used torch.bmmandtorch.matmul (only the ones I just came up with) as the method of calculating the product of tensors in pytorch`.
――However, the dimension of the tensor is predetermined in the product calculation method mentioned above. (For example, 3D tensor x 2D tensor, 2D tensor x 1D tensor, etc.)
――In this case, it was difficult to check the document one by one and use the function (method) according to the dimension.
Arithmetic that tends to be deep learning
import torch as t
X = t.rand(3,10,5)
Y = t.rand(3,20,5)
--Mini batch is $ 3 $ --The size of matrix X is $ 10 \ times5 $ --The size of matrix Y is $ 20 \ times5 $
In such a case, I want to calculate the product of each matrix in the mini-batch and calculate the matrix with the size of $ 10 \ times20 $. And you may want to return it as a batch.
--In other words, I want you to return the tensor $ 3 \ times10 \ times20 $ as the result of the XY operation. --As a result, you can calculate as follows
t.einsum('bnm,bkm->bnk',X,Y).size()
>> torch.Size([3, 10, 20])
--By the way, when you want to return $ 3 \ times20 \ times10 $ instead of $ 3 \ times10 \ times20 $ --'bnm, bkm-> bkn' Pay attention to the difference here
t.einsum('bnm,bkm->bkn',X,Y).size()
>> torch.Size([3, 20, 10])
You can also calculate the product of a matrix and a vector. simply.
X = t.rand(3,10,5)
Y = t.rand(3,5)
t.einsum('bnm,bm->bn',X,Y).size()
>> torch.Size([3, 10])
――By using ʻeinsum, you can express the product of complicated tensors very easily. This is goodbye to those who forcibly matched the dimensions with torch.transpose, torch.view, torch.squeeze`, etc.
Recommended Posts