PyTorch memo (dimension management)

Introduction

The Lord is a beginner in deep learning. If you make a mistake, please kindly let me know.

Definition of tensor

>>> import torch
>>> tensor = torch.randn(2, 3, 3)
tensor([[[ 1.5399, -0.8363,  0.3968],
         [ 0.0699,  1.1410,  0.7154],
         [ 0.4368,  0.9433, -0.8077]],

        [[ 1.1562, -1.3698,  0.6734],
         [-0.6762,  0.1539, -0.1286],
         [-0.4542,  0.3858, -1.6197]]])

Manipulate tensor dimension by dimension

Find the sum of the nth dimension (compress the dimension)

>>> sum_tensor = tensor.sum(2, keepdim=False)
tensor([[ 1.1004,  1.9262,  0.5725],
        [ 0.4599, -0.6509, -1.6881]])

torch.Size([2, 3])

Find the sum in the nth dimension (do not compress the dimension)

>>> sum_tensor = tensor.sum(2, keepdim=True)
tensor([[[ 1.1004],
         [ 1.9262],
         [ 0.5725]],

        [[ 0.4599],
         [-0.6509],
         [-1.6881]]])

torch.Size([2, 3, 1])

Extend the dimension

When using GPU, you can speed up by using the following

>>> tensor.sum(2, keepdim=True).expand([3, 2, 3, 1])

tensor([[[[ 1.1004],
          [ 1.9262],
          [ 0.5725]],

         [[ 0.4599],
          [-0.6509],
          [-1.6881]]],


        [[[ 1.1004],
          [ 1.9262],
          [ 0.5725]],

         [[ 0.4599],
          [-0.6509],
          [-1.6881]]],


        [[[ 1.1004],
          [ 1.9262],
          [ 0.5725]],

         [[ 0.4599],
          [-0.6509],
          [-1.6881]]]])

Summary

I think I will add it again if the Lord is in trouble