PyTorch's book was difficult to understand, so I supplemented it

Why did you write this article?

I bought this book because I wanted to learn about PyTorch.

• Can be used in the field! Introduction to PyTorch Development Creating a Deep Learning Model and Implementing It in an Application

After comparing some of them, I chose the one for beginners, but ... I don't know! !! !! It became . _ ** I haven't written the output, so I don't know what is output and what kind of viewpoint to check! !! ** _

So, I've put together a ** supplement ** that says this is probably what I'm trying to say. The execution environment is Google Colaboratory.

As a preliminary preparation, install PyTorch.

IN

import torch

PyTorch uses the GPU. Let's check if the GPU is available.

IN

#Check if GPU is available
print(torch.cuda.is_available())

OUT

True

What is Tensor?

According to the book, Tensor is

--A data structure for handling multidimensional arrays --Has almost the same API as Numpy's ndarray --Supports calculations on GPU

That's right.

In short, it's a PyTorch multidimensional array. In the book, there were explanations and sample code of mess and data types, but I just defined it ** I have not confirmed the contents at all **. So, ** I will check the data **.

IN

#Create by passing a nested list
t = torch.tensor([[1, 2], [3, 4.]])
print("dtype: {}\ndevice: {}".format(t.dtype, t.device))
t

OUT

dtype: torch.float32
device: cpu
tensor([[1., 2.],
        [3., 4.]])

dtype is the data type and device is whether the Tensor is on the CPU or GPU. Let's make a Tensor on the GPU.

IN

t = torch.tensor([[1, 2], [2, 4.]], device="cuda:0")
print("dtype: {}\ndevice: {}".format(t.dtype, t.device))
t

OUT

dtype: torch.float32
device: cuda:0
tensor([[1., 2.],
        [2., 4.]], device='cuda:0')

It seems that you can also create a Tensor on the GPU with torch.cuda.FloatTensor.

IN

t = torch.cuda.FloatTensor([[1, 2], [2, 4.]])
print("dtype: {}\ndevice: {}".format(t.dtype, t.device))
t

dtype: torch.float32
device: cuda:0
tensor([[1., 2.],
        [2., 4.]], device='cuda:0')

By the way, cuda is according to Wikipedia

CUDA (Compute Unified Device Architecture) is a general-purpose parallel computing platform (parallel computing architecture) and programming model for GPUs developed and provided by NVIDIA.

That's right.

Now let's change the data type.

IN

#Specify 64-bit data type
t = torch.tensor([[1, 2], [2, 4.]], dtype=torch.float64)
print("dtype: {}\ndevice: {}".format(t.dtype, t.device))
t

OUT

dtype: torch.float64
device: cpu
tensor([[1., 2.],
        [2., 4.]], dtype=torch.float64)

A 64-bit signed integer can also be defined with torch.LongTensor.

IN

t = torch.LongTensor([[1, 2], [2, 4.]])
print("dtype: {}\ndevice: {}".format(t.dtype, t.device))
t

OUT

dtype: torch.int64
device: cpu
tensor([[1, 2],
        [2, 4]])

The data types are summarized in this document. https://pytorch.org/docs/stable/tensors.html

Try transferring the Tensor created on the CPU to the GPU.

IN

t = torch.zeros(100, 10).to("cuda:0")
print("dtype: {}\ndevice: {}".format(t.dtype, t.device))
t[:2] #Slices are also available in Tensor

OUT

dtype: torch.float32
device: cuda:0
tensor([[0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]], device='cuda:0')

If you want to see the shape of the Tensor, use the size function instead of the shape function. This is different from NumPy.

IN

#shape is obtained by size
# t.shape() #Compile error
t.size()

OUT

torch.Size([100, 10])

Use the numpy function to convert a tensor to a Numpy ndarray.

IN

t = torch.tensor([[1, 2], [3, 4.]])
x = t.numpy()
print("type: {}".format(type(x)))
x

OUT

type: <class 'numpy.ndarray'> 
array([[1., 2.],
       [3., 4.]], dtype=float32)

The Tensor on the GPU needs to be converted to a CPU.

IN

t = torch.cuda.FloatTensor([[1, 2], [3, 4.]])
x = t.to("cpu").numpy()
print("type: {}".format(type(x)))
x

OUT

type: <class 'numpy.ndarray'>
array([[1., 2.],
       [3., 4.]], dtype=float32)

Tensor conversion

Let's try a confusing way to convert a tensor.

Reshape in Numpy is executed by view function.

IN

# 2 *2 to 4*Change to 1
#view is similar to reshape of ndarray
x1 = torch.tensor([[1, 2], [3, 4.]])
x1.view(4, 1)

OUT

tensor([[1.],
        [2.],
        [3.],
        [4.]])

There are several ways to write transpose.

IN

x2 = torch.tensor([[10, 20, 30], [40, 50, 60.]])
x2.T

OUT

tensor([[10., 40.],
        [20., 50.],
        [30., 60.]])

IN

x2.t()

OUT

tensor([[10., 40.],
        [20., 50.],
        [30., 60.]])

Transpose can also be transposed, but in addition to transposing, it can also be used to sort the image data format from HWC (vertical, horizontal, color) to CHW (color, horizontal, vertical).

IN

hwc_img_data = torch.rand(1, 5, 4, 3)
hwc_img_data

OUT

tensor([[[[0.9248, 0.7545, 0.5603],
          [0.5339, 0.6627, 0.7652],
          [0.7082, 0.5146, 0.9273],
          [0.8437, 0.7064, 0.1053]],

         [[0.2080, 0.8018, 0.6833],
          [0.5892, 0.9264, 0.9315],
          [0.0872, 0.1898, 0.5745],
          [0.2192, 0.1187, 0.7537]],

         [[0.9680, 0.9239, 0.8698],
          [0.2339, 0.9918, 0.3446],
          [0.6669, 0.4148, 0.2037],
          [0.1055, 0.0353, 0.3679]],

         [[0.7079, 0.4069, 0.1181],
          [0.1983, 0.0452, 0.5788],
          [0.6378, 0.7050, 0.1148],           
          [0.3960, 0.1924, 0.2714]],

         [[0.3127, 0.1320, 0.7232],
          [0.3484, 0.7617, 0.4725],
          [0.4863, 0.9178, 0.3092],
          [0.6279, 0.4259, 0.3828]]]])

I somehow understood that I would change from 1 * 5 * 4 * 3 to 1 * 4 * 5 * 3 (5 and 4 would be interchanged).

IN

# 1 * 5 * 4 *3 to 1* 4 * 5 *Change to 3
hwc_img_data.transpose(1, 2)

OUT

tensor([[[[0.9248, 0.7545, 0.5603],
          [0.2080, 0.8018, 0.6833],
          [0.9680, 0.9239, 0.8698],
          [0.7079, 0.4069, 0.1181],
          [0.3127, 0.1320, 0.7232]],

         [[0.5339, 0.6627, 0.7652],
          [0.5892, 0.9264, 0.9315],
          [0.2339, 0.9918, 0.3446],
          [0.1983, 0.0452, 0.5788],
          [0.3484, 0.7617, 0.4725]],

         [[0.7082, 0.5146, 0.9273],
          [0.0872, 0.1898, 0.5745],
          [0.6669, 0.4148, 0.2037],
          [0.6378, 0.7050, 0.1148],
          [0.4863, 0.9178, 0.3092]],

         [[0.8437, 0.7064, 0.1053],
          [0.2192, 0.1187, 0.7537],
          [0.1055, 0.0353, 0.3679],
          [0.3960, 0.1924, 0.2714],
          [0.6279, 0.4259, 0.3828]]]])

IN

#1 more* 4 * 5 *3 to 1* 3 * 4 *Change to 5
#Now you can convert from hwc to cwh
hwc_img_data.transpose(1, 2).transpose(1, 3)

OUT

tensor([[[[0.9248, 0.5339, 0.7082, 0.8437],
          [0.2080, 0.5892, 0.0872, 0.2192],
          [0.9680, 0.2339, 0.6669, 0.1055],
          [0.7079, 0.1983, 0.6378, 0.3960],
          [0.3127, 0.3484, 0.4863, 0.6279]],

         [[0.7545, 0.6627, 0.5146, 0.7064],
          [0.8018, 0.9264, 0.1898, 0.1187],
          [0.9239, 0.9918, 0.4148, 0.0353],
          [0.4069, 0.0452, 0.7050, 0.1924],
          [0.1320, 0.7617, 0.9178, 0.4259]],

         [[0.5603, 0.7652, 0.9273, 0.1053],
          [0.6833, 0.9315, 0.5745, 0.7537],
          [0.8698, 0.3446, 0.2037, 0.3679],
          [0.1181, 0.5788, 0.1148, 0.2714],
          [0.7232, 0.4725, 0.3092, 0.3828]]]])

Tensor and automatic differentiation

If the requires_grad attribute is set to True, automatic differentiation will be performed. The book doesn't describe in detail where and what will appear, so I'll add it.

IN

x = torch.randn(100, 3)
a = torch.tensor([1, 2, 3.], requires_grad=True)

y = torch.mv(x, a)
print(y.grad_fn) #Make sure that the calculation graph for calculating the gradient is stored

o = y.sum()

#Get the gradient
#Variables that perform backwards must be scalar
o.backward()

#Automatic differentiation is performed, a.Gradient is acquired in grad
print(a.grad)

a.grad != x.sum(0)

OUT

<MvBackward object at 0x7f9c7998e3c8>
tensor([ 2.5016,  6.3925, -6.3674])
tensor([False, False, False])