Summary of modules and classes in Python-TensorFlow2-

Introduction

We've summarized the relationships between Python packages, modules, classes, methods, functions, and instances. To summarize, I've introduced a TensorFlow 2.0 tutorial as an example.

Who is the target of this article

The target audience for this article is: -I want to check the relationships between Python packages, modules, classes, methods, functions, and instances. ・ I'm new to Python and don't understand the structure of libraries such as TensorFlow.

Rough content of this article

  1. The Tens of Flow tutorial used in this article

  2. First, import the library

  3. Structure of TensorFlow (general library)

  4. Examples of packages, modules and functions

  5. What are classes, methods and instances?

  6. Examples of classes, methods and instances

1. The Tens of Flow tutorial used in this article

In this article, we will use the following TensorFlow 2.0 tutorial as an example. TensorFlow Tutorial The full code is below.

TensorFlow tutorial code


from __future__ import absolute_import, division, print_function, unicode_literals
import tensorflow as tf
mnist = tf.keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0
model = tf.keras.models.Sequential([
  tf.keras.layers.Flatten(input_shape=(28, 28)),
  tf.keras.layers.Dense(128, activation='relu'),
  tf.keras.layers.Dropout(0.2),
  tf.keras.layers.Dense(10, activation='softmax')
])
model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])
model.fit(x_train, y_train, epochs=5)
model.evaluate(x_test,  y_test, verbose=2)

2. First, import the library

To use a library (such as TensorFlow), you need to import the library. Also, before importing the library, it is necessary to install it in advance (pip install etc.), but since there are already many articles on this area, I will omit it. Below is the import part of TensorFlow excerpted from the tutorial.

import part


import tensorflow as tf

What this is doing is that I want to use the program code to import (tensorflow in this case) in the program I execute (the tutorial code shown in Chapter 1), so prepare (execute in advance). It is an image that tells you to take it. Also, although I write as tf here, it is troublesome to type tensorflow every time I use the TensorFlow function, so I just declare that I will abbreviate it as tf from the next time.

3. Structure of TensorFlow (general software library)

Not limited to TensorFlow, software libraries are composed of the following elements. -Package: A code group in which multiple different "modules" are collected is called a "package". -Module: A .py file (python program code) that contains "classes" and "functions". -Class: A group of "methods" is called a "class". -Method: A function defined in a "class". The difference from a normal function will be explained at the bottom of the article. -Function: A summary of some processing. For example, a function that adds 100 times or a function that calculates the volume from the length of three sides of a box. -Instance: The object to be actually processed is given to the class. It is explained in detail below.

If you read only the above, you will not understand it, so please take a look with the figure below. You can see the following two points.

図1.png

  1. We usually make it easier to manage files in a hierarchical structure (creating folders inside folders). Similarly, packages and modules manage functions and methods (although methods are bundled with classes) that are part of the actual data processing in an easy-to-understand manner.
  2. The actual data processing part is described in Functions and Methods.

4. Examples of packages, modules and functions

As an example, let's show a function called load_data of TensorFlow in the following figure in a hierarchical structure. The figures on the left and right show the same thing.

図2.png

As you can see from the (example) in the above figure, load_data is a function in mnist (module) in datasets (module) in keras (module) in tensorflow (package). And in the Python program code, a. (Dot) indicates that you are going down the hierarchy.

So, the code for loading the machine learning dataset below, excerpted from the tutorial, First, assign the module mnist in datasets in keras in tensorflow to the name mnist. In other words, we are creating a new name, mnist, and assigning mnist (module) in tensorflow. In the second line, the mnist created in the first line is added with. (Dot), and a function called load_data () in the lower hierarchy is used. By the way, () means that nothing is given as an argument.

Data load part


mnist = tf.keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()

So the above code has exactly the same meaning as the code below.

Data load part (when mnist declaration is omitted)


(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()

You can easily check the structure of modules and functions in tensorflow as shown in the above figure on the official website below. * HP is version 2.1. TensorFlow Core v2.1.0 Module: tf.keras.datasets

If you open the tf.keras.datasets page with the link above, you can get the information as shown below. The numbers correspond to the figures below. ➀ tensorflow Hierarchical information of all packages and modules is displayed in a list. ➁ The currently open tf.keras.datasets are shown to be modules. ➂ It shows what modules are in the hierarchy one level below the currently open tf.keras.datasets.

図3.png

5. What are classes, methods and instances?

Classes, methods, and instances are inevitable items for learning about Python and for understanding TensorFlow. I feel that packages, modules, and functions are sufficient for the purpose of managing the processing of a program in an easy-to-read manner, but classes exist to realize the idea of object-oriented programming in Python. There are many articles about object-oriented programming, so please have a look there. Here, I will explain only the difference between how to use classes, methods and instances, and functions.

First, the role of a method is almost the same as that of a function, and it does some processing together. The only major difference is that, as shown in the figure below, functions are used alone, while methods are used as a set with a class. In order to use the method, it is necessary to create something like a box from the class to the instance, which stores the target to be processed by the method and the data after processing, as shown in the above figure. Functions, on the other hand, can perform operations directly on ordinary variables.

図4.png

It looks like this when written in code. It may be difficult to understand the class at first glance, so please check the explanation article of the class.

Create an instance from a class and execute a method


Instance = Class(Class arguments) # クラスからインスタンスを作成しています。Class argumentsが空()Often.
Instance.method(Method arguments) #The method is being processed for the created instance.

Therefore, in order to use the method, it is necessary to create an instance from the class in advance (unlike the function), so the following code will result in an error. This is a big difference from a function. * There are special methods, but I think it's okay if you don't know them at first.

Execute method without creating an instance


Class.method(Method arguments) #An error will occur.

6. Examples of classes, methods and instances

As an example, the following figure shows the structure of the TensorFlow class Sequential and the methods compile, fit, and evaluate in the hierarchy below it. The figures on the left and right show the same thing.

図5.png

And, in the code of the following data loading part excerpted from the tutorial, Sequential is a class, compile, fit, evaluate is a method, and model is Sequential is an instance. First, create an instance called model using a class called Sequential, and for this instance, use the methods compile (definition of learning method), fit (model learning), and evaluate (model evaluation) in the hierarchy below the class called Sequential. We are processing in order. * By the way, tf.keras.layers.Flatten etc. in the list of arguments of Sequential are classes and are used to define the structure of the machine learning model.

Examples of classes and instances in the TensorFlow tutorial


model = tf.keras.models.Sequential([
  tf.keras.layers.Flatten(input_shape=(28, 28)),
  tf.keras.layers.Dense(128, activation='relu'),
  tf.keras.layers.Dropout(0.2),
  tf.keras.layers.Dense(10, activation='softmax')
])
model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])
model.fit(x_train, y_train, epochs=5)
model.evaluate(x_test,  y_test, verbose=2)

You can check the structure of tensorflow classes and methods and their arguments from the official website below as well as the functions. TensorFlow Core v2.1.0 Class Method: tf.keras.model.Sequential

At the end

Using TensorFlow as an example, I tried to organize the library very roughly. I hope this article will help you in studying Python. * If you make a mistake, please let us know in the comments.

References

TensorFlow TensorFlow Tutorial Python tutorial

Recommended Posts

Summary of modules and classes in Python-TensorFlow2-
Summary of OSS tools and libraries created in 2016
Summary of date processing in Python (datetime and dateutil)
List method argument information for classes and modules in Python
Summary of various operations in Tensorflow
Summary of Python indexes and slices
Modules and packages in Python are "namespaces"
Summary of methods often used in pandas
Summary of various for statements in Python
Summary of stumbling blocks in installing CaboCha
Separation of design and data in matplotlib
Summary of built-in methods in Python list
Project Euler # 1 "Multiples of 3 and 5" in Python
Organize python modules and packages in a mess
Correspondence summary of array operation of ruby and python
Summary of the differences between PHP and Python
Summary of how to import files in Python 3
Organize the meaning of methods, classes and objects
Summary of tools used in Command Line vol.8
Summary of how to use MNIST in Python
Installation of Python3 and Flask [Environment construction summary]
Summary of tools used in Command Line vol.5
Header shifts in read_csv () and read_table () of Pandas
Summary of evaluation functions used in machine learning
Coexistence of Anaconda 2 and Anaconda 3 in Jupyter + Bonus (Julia)
I / O related summary of python and fortran
Explanation of edit distance and implementation in Python
Summary of pickle and unpickle processing of user-defined class
Summary of Excel operations using OpenPyXL in Python
[Introduction to Python] Summary of functions and methods that frequently appear in Python [Problem format]
"Linear regression" and "Probabilistic version of linear regression" in Python "Bayesian linear regression"
The simplest Python memo in Japan (classes and objects)
Summary of tools needed to analyze data in Python
Full-width and half-width processing of CSV data in Python
About import error of numpy and scipy in anaconda
Calculation of standard deviation and correlation coefficient in Python
Summary of Linux (UNIX) commands that appeared in Progate
Python --Explanation and usage summary of the top 24 packages
[Python] Type Error: Summary of error causes and remedies for'None Type'
[python] Get the list of classes defined in the module
Difference between Ruby and Python in terms of variables
[Competition Pro] Summary of stock buying and selling problems
Overview of generalized linear models and implementation in Python
Get an abstract understanding of Python modules and packages
Sample of getting module name and class name in Python
Numerical summary of data
Nest modules in Boost.Python
List of python modules
Multiple inheritance of classes
Summary of Tensorflow / Keras
Summary of pyenv usage
Python packages and modules
Summary of string operations
Dynamic loading of modules
Summary of Python arguments
Summary of logrotate software logrotate
Summary of test method
Summary of Prototype patterns introductory design patterns learned in Java language
Basic summary of data manipulation in Python Pandas-Second half: Data aggregation
Hot backup and restore of virtual machines in Hyper-V Server 2019
Types of preprocessing in natural language processing and their power