PyTorch's book was difficult to understand, so I supplemented it

Why did you write this article?

I bought this book because I wanted to learn about PyTorch.

After comparing some of them, I chose the one for beginners, but ... I don't know! !! !! It became . _ ** I haven't written the output, so I don't know what is output and what kind of viewpoint to check! !! ** _

So, I've put together a ** supplement ** that says this is probably what I'm trying to say. The execution environment is Google Colaboratory.

As a preliminary preparation, install PyTorch.

IN


import torch

PyTorch uses the GPU. Let's check if the GPU is available.

IN


#Check if GPU is available
print(torch.cuda.is_available())

OUT


True

What is Tensor?

According to the book, Tensor is

--A data structure for handling multidimensional arrays --Has almost the same API as Numpy's ndarray --Supports calculations on GPU

That's right.

In short, it's a PyTorch multidimensional array. In the book, there were explanations and sample code of mess and data types, but I just defined it ** I have not confirmed the contents at all **. So, ** I will check the data **.

IN


#Create by passing a nested list
t = torch.tensor([[1, 2], [3, 4.]])
print("dtype: {}\ndevice: {}".format(t.dtype, t.device))
t

OUT


dtype: torch.float32
device: cpu
tensor([[1., 2.],
        [3., 4.]])

dtype is the data type and device is whether the Tensor is on the CPU or GPU. Let's make a Tensor on the GPU.

IN


t = torch.tensor([[1, 2], [2, 4.]], device="cuda:0")
print("dtype: {}\ndevice: {}".format(t.dtype, t.device))
t

OUT


dtype: torch.float32
device: cuda:0
tensor([[1., 2.],
        [2., 4.]], device='cuda:0')

It seems that you can also create a Tensor on the GPU with torch.cuda.FloatTensor.

IN


t = torch.cuda.FloatTensor([[1, 2], [2, 4.]])
print("dtype: {}\ndevice: {}".format(t.dtype, t.device))
t
dtype: torch.float32
device: cuda:0
tensor([[1., 2.],
        [2., 4.]], device='cuda:0')

By the way, cuda is according to Wikipedia

CUDA (Compute Unified Device Architecture) is a general-purpose parallel computing platform (parallel computing architecture) and programming model for GPUs developed and provided by NVIDIA.

That's right.

Now let's change the data type.

IN


#Specify 64-bit data type
t = torch.tensor([[1, 2], [2, 4.]], dtype=torch.float64)
print("dtype: {}\ndevice: {}".format(t.dtype, t.device))
t

OUT


dtype: torch.float64
device: cpu
tensor([[1., 2.],
        [2., 4.]], dtype=torch.float64)

A 64-bit signed integer can also be defined with torch.LongTensor.

IN


t = torch.LongTensor([[1, 2], [2, 4.]])
print("dtype: {}\ndevice: {}".format(t.dtype, t.device))
t

OUT


dtype: torch.int64
device: cpu
tensor([[1, 2],
        [2, 4]])

The data types are summarized in this document. https://pytorch.org/docs/stable/tensors.html

Try transferring the Tensor created on the CPU to the GPU.

IN


t = torch.zeros(100, 10).to("cuda:0")
print("dtype: {}\ndevice: {}".format(t.dtype, t.device))
t[:2] #Slices are also available in Tensor

OUT


dtype: torch.float32
device: cuda:0
tensor([[0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]], device='cuda:0')

If you want to see the shape of the Tensor, use the size function instead of the shape function. This is different from NumPy.

IN


#shape is obtained by size
# t.shape() #Compile error
t.size()

OUT


torch.Size([100, 10])

Use the numpy function to convert a tensor to a Numpy ndarray.

IN


t = torch.tensor([[1, 2], [3, 4.]])
x = t.numpy()
print("type: {}".format(type(x)))
x

OUT


type: <class 'numpy.ndarray'> 
array([[1., 2.],
       [3., 4.]], dtype=float32)

The Tensor on the GPU needs to be converted to a CPU.

IN


t = torch.cuda.FloatTensor([[1, 2], [3, 4.]])
x = t.to("cpu").numpy()
print("type: {}".format(type(x)))
x

OUT


type: <class 'numpy.ndarray'>
array([[1., 2.],
       [3., 4.]], dtype=float32)

Tensor conversion

Let's try a confusing way to convert a tensor.

Reshape in Numpy is executed by view function.

IN


# 2 *2 to 4*Change to 1
#view is similar to reshape of ndarray
x1 = torch.tensor([[1, 2], [3, 4.]])
x1.view(4, 1)

OUT


tensor([[1.],
        [2.],
        [3.],
        [4.]])

There are several ways to write transpose.

IN


x2 = torch.tensor([[10, 20, 30], [40, 50, 60.]])
x2.T

OUT


tensor([[10., 40.],
        [20., 50.],
        [30., 60.]])

IN


x2.t()

OUT


tensor([[10., 40.],
        [20., 50.],
        [30., 60.]])

Transpose can also be transposed, but in addition to transposing, it can also be used to sort the image data format from HWC (vertical, horizontal, color) to CHW (color, horizontal, vertical).

IN


hwc_img_data = torch.rand(1, 5, 4, 3)
hwc_img_data

OUT


tensor([[[[0.9248, 0.7545, 0.5603],
          [0.5339, 0.6627, 0.7652],
          [0.7082, 0.5146, 0.9273],
          [0.8437, 0.7064, 0.1053]],

         [[0.2080, 0.8018, 0.6833],
          [0.5892, 0.9264, 0.9315],
          [0.0872, 0.1898, 0.5745],
          [0.2192, 0.1187, 0.7537]],

         [[0.9680, 0.9239, 0.8698],
          [0.2339, 0.9918, 0.3446],
          [0.6669, 0.4148, 0.2037],
          [0.1055, 0.0353, 0.3679]],

         [[0.7079, 0.4069, 0.1181],
          [0.1983, 0.0452, 0.5788],
          [0.6378, 0.7050, 0.1148],           
          [0.3960, 0.1924, 0.2714]],

         [[0.3127, 0.1320, 0.7232],
          [0.3484, 0.7617, 0.4725],
          [0.4863, 0.9178, 0.3092],
          [0.6279, 0.4259, 0.3828]]]])

I somehow understood that I would change from 1 * 5 * 4 * 3 to 1 * 4 * 5 * 3 (5 and 4 would be interchanged).

IN


# 1 * 5 * 4 *3 to 1* 4 * 5 *Change to 3
hwc_img_data.transpose(1, 2)

OUT


tensor([[[[0.9248, 0.7545, 0.5603],
          [0.2080, 0.8018, 0.6833],
          [0.9680, 0.9239, 0.8698],
          [0.7079, 0.4069, 0.1181],
          [0.3127, 0.1320, 0.7232]],

         [[0.5339, 0.6627, 0.7652],
          [0.5892, 0.9264, 0.9315],
          [0.2339, 0.9918, 0.3446],
          [0.1983, 0.0452, 0.5788],
          [0.3484, 0.7617, 0.4725]],

         [[0.7082, 0.5146, 0.9273],
          [0.0872, 0.1898, 0.5745],
          [0.6669, 0.4148, 0.2037],
          [0.6378, 0.7050, 0.1148],
          [0.4863, 0.9178, 0.3092]],

         [[0.8437, 0.7064, 0.1053],
          [0.2192, 0.1187, 0.7537],
          [0.1055, 0.0353, 0.3679],
          [0.3960, 0.1924, 0.2714],
          [0.6279, 0.4259, 0.3828]]]])

IN


#1 more* 4 * 5 *3 to 1* 3 * 4 *Change to 5
#Now you can convert from hwc to cwh
hwc_img_data.transpose(1, 2).transpose(1, 3)

OUT


tensor([[[[0.9248, 0.5339, 0.7082, 0.8437],
          [0.2080, 0.5892, 0.0872, 0.2192],
          [0.9680, 0.2339, 0.6669, 0.1055],
          [0.7079, 0.1983, 0.6378, 0.3960],
          [0.3127, 0.3484, 0.4863, 0.6279]],

         [[0.7545, 0.6627, 0.5146, 0.7064],
          [0.8018, 0.9264, 0.1898, 0.1187],
          [0.9239, 0.9918, 0.4148, 0.0353],
          [0.4069, 0.0452, 0.7050, 0.1924],
          [0.1320, 0.7617, 0.9178, 0.4259]],

         [[0.5603, 0.7652, 0.9273, 0.1053],
          [0.6833, 0.9315, 0.5745, 0.7537],
          [0.8698, 0.3446, 0.2037, 0.3679],
          [0.1181, 0.5788, 0.1148, 0.2714],
          [0.7232, 0.4725, 0.3092, 0.3828]]]])

Tensor and automatic differentiation

If the requires_grad attribute is set to True, automatic differentiation will be performed. The book doesn't describe in detail where and what will appear, so I'll add it.

IN


x = torch.randn(100, 3)
a = torch.tensor([1, 2, 3.], requires_grad=True)

y = torch.mv(x, a)
print(y.grad_fn) #Make sure that the calculation graph for calculating the gradient is stored

o = y.sum()

#Get the gradient
#Variables that perform backwards must be scalar
o.backward()

#Automatic differentiation is performed, a.Gradient is acquired in grad
print(a.grad)

a.grad != x.sum(0)

OUT


<MvBackward object at 0x7f9c7998e3c8>
tensor([ 2.5016,  6.3925, -6.3674])
tensor([False, False, False])

Recommended Posts

PyTorch's book was difficult to understand, so I supplemented it
[Introduction to json] No, I was addicted to it. .. .. ♬
I was able to mock AWS-Batch with python, moto, so I will leave it
Docker x visualization didn't work and I was addicted to it, so I summarized it!
I didn't understand the Resize of TensorFlow so I tried to summarize it visually.
I set up TensowFlow and was addicted to it, so make a note
I was able to repeat it in Python: lambda
[IOS] GIF animation with Pythonista3. I was addicted to it.
It was difficult to understand kernel tricks in support vector machines (Part 2: Kernel regression, ridge regression, etc.)
[Qualification] I passed LinuC Level 1, so I will write about how to study and how it was.
I wrote it in Go to understand the SOLID principle
There was a doppelganger, so I tried to distinguish it with artificial intelligence (laughs) (Part 1)
I couldn't understand Fence Repair of the ant book easily, so I will follow it in detail.
I was addicted to multiprocessing + psycopg2
I didn't understand the behavior of numpy's argsort, so I will summarize it.
I tried to make a calculator with Tkinter so I will write it
I installed PySide2, but pyside2-uic didn't work, so I managed to do it.
Use Python from Java with Jython. I was also addicted to it.
(Beginner) I don't understand SSH and Linux command ssh-keygen, so I summarized it.
I was addicted to trying Cython with PyCharm, so make a note
The tree.plot_tree of scikit-learn was very easy and convenient, so I tried to summarize how to use it easily.
I automated job can stamping with selenium and deployed it to Google Cloud Functions, but it was quite difficult
In IPython, when I tried to see the value, it was a generator, so I came up with it when I was frustrated.
Wrangle x Python book I tried it [2]
I was addicted to pip install mysqlclient
Binary tree traverse is difficult to understand
I was addicted to Flask on dotCloud
I researched Docker, so I will summarize it
Wrangle x Python book I tried it [1]
I studied about Linux, so I summarized it.
What I was addicted to Python autorun
I tried to publish my own module so that I can pip install it
When I investigated whether the COTOHA API could understand comics, it was reasonable.
Somehow the code I wrote worked and I was impressed, so I will post it
P100-PCIE-16GB was added to the GPU of Google Colab before I knew it