I enjoyed writing TypeScript for about a year, but I started to use Python 3 for business about a month ago. Python is simple and fun, but I've inherited it from others, and it's hard to develop and have no type.
The hardest part is code reading, what does this function return? Do not know at a glance what is contained in this variable. …… There is no type as a document. If it doesn't exist, I decided to introduce it, so I decided to introduce type annotation and mypy.
Even if you suddenly introduce the mold to the ruggedness, it will be painful on the contrary, so I took the method of introducing the mold without overdoing it from the state without the mold, and I was able to introduce it quite well, so this time the procedure at that time and the knowledge obtained , I would like to introduce some tips.
This time I'm testing in a pipenv environment, but I don't think there is any particular difference in mypy with pip etc.
python3.7.5 pipenv, version 2018.11.26
Please refer to here for the introduction of pipenv https://pipenv-ja.readthedocs.io/ja/translate-ja/index.html
Install mypy under the project directory.
cd ./myproject
pipenv install mypy -d
If mypy is installed in global pip and you want to statically typecheck the code under the src directory, you can typecheck with mypy ./src, but if mypy is only in pipenv, you will get an error. Become.
$ mypy ./src
Command 'mypy' not found, but can be installed with:
sudo apt install mypy
If you enter the virtual environment of pipenv, you can execute it without any problem.
$ pipenv shell
(myproject) $ mypy ./src
Success: no issues found in 2 source files
It is troublesome to enter the virtual environment every time, so let's register the script in Pipfile.
[scripts]
type-check = "mypy ./src"
Reference https://pipenv-ja.readthedocs.io/ja/translate-ja/advanced.html#custom-script-shortcuts
Since the command executed by the script is executed in the environment of pipenv, mypy can be executed without executing pipenv shell.
$ pipenv run type-check
mypy.ini: No [mypy] section in config file
Success: no issues found in 1 source files
Use this command when performing type checking, such as in CI.
I think most people know it, but let's review the basic types. Of the official documents, I think that the following 8 types are used in the normal type inspection at most. https://docs.python.org/ja/3.7/library/stdtypes.html
If you get lost, you can put it in the built-in function type () and see the result, so you don't even have to remember it.
| Type of type | Model name | Example |
|---|---|---|
| Boolean type | bool | True |
| Integer type | int | 10 |
| Floating point type | float | 1.2 |
| Text sequence type (character string type) | str | 'hoge' |
| List type | list | [1, 2, 3] |
| Tuple type | tuple | ('a', 'b') |
| Dictionary type (mapping type) | dict | { 'a': 'hoge', 'b': 'fuga'} |
| Collective type | set | { 'j', 'k', 'l'} |
First, let's create an untyped function.
Create my_module.py under ./src.
my_module.py
def get_greeting(time):
if 4 <= time < 10:
return 'Good morning!'
elif 10 <= time < 14:
return 'Hello!'
elif 14 <= time < 24:
return 'Goog afternoon.'
elif 0 <= time < 4:
return 'zzz..'
else:
return ''
if __name__ == "__main__":
print(get_greeting('morning'))
I tried to make it a function that receives the time from 0 to 24 and returns a greeting. Now when I run a type check ...
$ pipenv run type-check
Success: no issues found in 1 source file
No error! The reason is that since type annotation is not performed, the return value and arguments of the function will be the basic Any type (any type). (If there is an existing code base, I think this is fine because it does not cause an error when introducing the type and it does not break my heart.) So next, let's add type annotation.
def get_greeting(time: int) -> str:
if 4 <= time < 10:
return 'Good morning!'
elif 10 <= time < 14:
return 'Hello!'
elif 14 <= time < 20:
return 'Goog afternoon.'
elif 0 <= time < 4:
return 'zzz..'
else:
return None
if __name__ == "__main__":
print(get_greeting('morning'))
Added a type annotation on the first line to indicate "receive an integer type and return a string type". Try running the type check again in this state.
$ pipenv run type-check
src/my_module.py:14: error: Argument 1 to "get_greeting" has incompatible type "str"; expected "int"
Found 1 error in 1 file (checked 1 source file)
This time I got an error properly. When I read the error message, I passed a string when calling the function get_greeting on line 14. If you execute it as it is, a runtime error will occur. If you change the code to pass an integer type and run the type check again, the error will disappear.
print(get_greeting(10))
By adding type annotations, we were able to make the code easier to understand and prevent run-time errors.
Even so, there are times when you want to force type annotations. In that case, create a configuration file.
mypy.ini
[mypy]
python_version = 3.7
disallow_untyped_calls = True
disallow_untyped_defs = True
Modify the script to specify the configuration file as well.
[scripts]
type-check = "mypy ./src --config-file ./mypy.ini"
By doing this, if you forget to add the type annotation, an error will be returned.
$ pipenv run type-check
src/my_module.py:1: error: Function is missing a type annotation
src/my_module.py:14: error: Call to untyped function "get_greeting" in typed context
Found 2 errors in 1 file (checked 1 source file)
Reference https://mypy.readthedocs.io/en/latest/config_file.html
Although it can be deployed into an existing code base with Any tolerance, it is inevitable that a large number of errors will occur during deployment if the original code base is large. Please wait a moment before your heart is about to break. 90% of the errors should disappear just by executing the following two.
Reference https://mypy.readthedocs.io/en/latest/existing_code.html#start-small
For example, if you have the following code:
import request
When I do a type check, I get an error of 3 lines.
$ pipenv run type-check
src/my_module.py:1: error: Cannot find implementation or library stub for module named 'request'
src/my_module.py:1: note: See https://mypy.readthedocs.io/en/latest/running_mypy.html#missing-imports
Found 1 error in 1 file (checked 1 source file)
This is because there is no type definition file (stub) for the imported module.
Since it is difficult to prepare all the type definition files at the time of installation, ** ignore ** in the settings of mypy.ini.
mypy.ini
[mypy-request.*]
ignore_missing_imports = True
This will ignore the lack of stubs in the import from request.
$ pipenv run type-check
Success: no issues found in 1 source file
Peace is now back.
I can't recommend it very much, but it's often used when you want to assign the wrong type in test etc.
Add a comment # type: ignore to the end of the line of code you want to ignore.
print(get_greeting('hoge')) #type: ignore
You can suppress the type error that should have occurred on this line.
You may say, "No, I want to use a stub." However, there is no guarantee that the developers of third-party modules have stubs available. It is troublesome to make it yourself. In such a case, let's generate it automatically.
You can automatically generate a stub by specifying a file or directory with the stubgen command, which can be used by inserting mypy.
$ stubgen foo.py bar.py
If it is an imported module, you can check the module path with * .__ path __, so you can also create a stub by directly specifying that path.
>>> import request
>>> request.__path__
['/home/username/.local/share/virtualenvs/myproject-xxxxxxxx/lib/python3.7/site-packages/request']
>>>
Once you know the path, run stubgen.
(myproject) $ stubgen /home/username/.local/share/virtualenvs/myproject-xxxxxxxx/lib/python3.7/site-packages/request
Processed 1 modules
Generated out/request/__init__.pyi
Running stubgen creates an out directory in the project root, so specify this path in mypy.ini so that mypy can see it.
mypy.ini
[mypy]
python_version = 3.7
mypy_path = ./out
Type inspection is now passed.
$ pipenv run type-check
Success: no issues found in 1 source file
The types generated by stubgen are not perfect. It will be almost Any type, so if you want to use it in earnest, you need to modify the stub file yourself.
Reference https://github.com/python/mypy/blob/master/docs/source/stubgen.rst
In addition to the built-in type, there are other types that are often used, so I will introduce them.
In mypy, except for built-in types, the typing module and the typing_extensions module call the classes of those types and use them.
It's a little different from typescript in terms of usability, but since most types including generic types are covered in these modules, it seems to be satisfying for those who want to do type programming.
Reference https://mypy.readthedocs.io/en/latest/
Optional
Functions such as returning an integer normally and returning None if an incorrect value is received are common. The return value in that case is int or None, but Optional can express this.
from typing import Optional
def sample(time: int) -> Optional[int]:
if 24 < time:
return None
else:
return time
List, Dict A list of integers, a list of character strings, etc. can be represented by List.
from typing import List
#List of integers
intList: List[int] = [1, 2, 3, 4]
#List of strings
strList: List[str] = ['a', 'b', 'c']
Similarly, if you use Dict, you can express something like "key is a character and value is an integer" even in a dictionary type.
from typing import Dict
#Dictionary type where key is a character and value is an integer
strIntDict: Dict[str, int] = {'a': 1, 'b': 2, 'c': 3, 'd': 4}
Union
You can create a union type that combines multiple people.
from typing import Union
strOrInt: Union[str, int] = 1 # OK
strOrInt = 'hoge' # OK
strOrInt = None # error: Incompatible types in assignment (expression has type "None", variable has type "Union[str, int]")
Any
Of course Any type is also possible if you do not want to specify the type
from typing import Any
string: str = 'hoge'
any: Any = string
any = 10 # OK
notAny = string
notAny = 10 # error: Incompatible types in assignment (expression has type "int", variable has type "str")
Callable
You can express the type of a function with Callable.
from typing import Callable
#Function type definition that takes one integer type argument and returns a string type
func: Callable[[int], str]
def sample(num: int) -> str:
return str(num)
func = sample
TypedDict
You may want to specify the value of a key in your map and specify what type of value the key has. If it is typescript, it is expressed by interface.
For example, suppose you have a dictionary-type value called movie.
movie = {'name': 'Blade Runner', 'year': 1982}
move has keys called name and year, but if you accidentally put an integer in name or put a character string in year when overwriting, it would be a problem. TypedDict makes it easy to express as a type.
from typing_extensions import TypedDict
Movie = TypedDict('Movie', {'name': str, 'year': int})
movie1: Movie = {'name': 'Blade Runner', 'year': 1982} # OK
movie2: Movie = {'name': 'Blade Runner', 'year': '1982'} # error: Incompatible types (expression has type "str", TypedDict item "year" has type "int")
It can also be expressed in the form of a class. Personally, I prefer this because it looks like a TS interface.
from typing_extensions import TypedDict
class Movie(TypedDict):
name: str
year: int
Please check the official document for details. https://mypy.readthedocs.io/en/latest/more_types.html#typeddict
What did you think? I hope you feel that starting a type in Python is a surprisingly low hurdle.
If you have a hard time with Python, let's introduce it now! It's more than I expected, so I can be happy.
The dissatisfaction is that the editor's support is not very solid. I'm using Pyright with a VS-Code extension, but I'd like to switch if there's something better.
Have a nice year-end!
Recommended Posts