A typed world that begins with Python

Starting with Python 3.5, a feature called Type Hints has been introduced.

This is a specification that allows you to add type annotations (type annotations), and specifically looks like the following (Abstract Quoted from 0484 / # abstract)).

def greeting(name: str) -> str:
    return 'Hello ' + name

The following part is actually annotating.

name: str: Annotate that the argument name is of type str
-> str: Annotate that the return type of the function greeting is str

Type Hints also mentions Type Comments in Variable Declarations (https://www.python.org/dev/peps/pep-0484/#type-comments).

x = []   # type: List[Employee]

This is really an extension of the comment, not the syntax, but if you're already commenting on these types, you can hijack the above notation and in the future you'll be able to type-check with some tool. There is a possibility.

This is the typed world that was introduced in Python.

Note that the added annotation is not checked at runtime. In short, it is an extension of the comment. Therefore, it is not enforceable, but it does nothing at runtime and does not affect performance.

Therefore, in principle, the syntax is for static analysis, but annotation information in typing.get_type_hints I think it is possible to implement run-time checking on your own (for example, type annotations).

Annotation notation

I would like to introduce the annotation notation along with typing.

Type aliases

It is possible to define type aliases.

Type aliases

Vector = List[float]

However, in my personal experience, this can be confusing (it's hard to tell if it's a class or an alias), so be careful about where you use it and how you name it.

Callable

This is a summary of function argument / return type definitions. It is used when a function is an argument like a callback function, or when a function is returned.

The notation is Callable ([argument type], return type). The following is an example of annotating feeder / async_query that takes a function as an argument with Callable.

Callable

from typing import Callable

def feeder(get_next_item: Callable[[], str]) -> None:
    # Body

def async_query(on_success: Callable[[int], None],
                on_error: Callable[[int, Exception], None]) -> None:
    # Body

Generics

So-called Generics are also available. You can write List <T> in Java etc. as follows.

Generics

from typing import Sequence, TypeVar

T = TypeVar('T')      # Declare type variable

def first(l: Sequence[T]) -> T:   # Generic function
    return l[0]

You can also limit the valid types of Generics with TypeVar. In the following, only str and bytes are allowed as AnyStr.

pep-0484/#generics

from typing import TypeVar

AnyStr = TypeVar('AnyStr', str, bytes)

def concat(x: AnyStr, y: AnyStr) -> AnyStr:
    return x + y

User-defined generic types

In the so-called Generics class, if you want to do something like MyClass <T>, define it as follows.

User-defined generic types

from typing import TypeVar, Generic

T = TypeVar('T')

class LoggedVar(Generic[T]):
    def __init__(self, value: T, name: str, logger: Logger) -> None:
        self.name = name
        self.logger = logger
        self.value = value

    def set(self, new: T) -> None:
        self.log('Set ' + repr(self.value))
        self.value = new

    def get(self) -> T:
        self.log('Get ' + repr(self.value))
        return self.value

    def log(self, message: str) -> None:
        self.logger.info('{}: {}'.format(self.name message))

Now you can use it like this:

from typing import Iterable

def zero_all_vars(vars: Iterable[LoggedVar[int]]) -> None:
    for var in vars:
        var.set(0)

The Any type

All types are subtypes of ʻAny. Note that ʻAny is different from ʻobject` in the class.

Union types

It is a collection of several acceptable types. ʻOptional is a kind of this ʻUnion.

Union types

from typing import Union

def handle_employee(e: Union[Employee, None]) -> None: ...

This is synonymous with:

from typing import Optional

def handle_employee(e: Optional[Employee]) -> None: ...

Method for checking

As mentioned above, I think the type annotation notation covers most of the scenes you want to type. However, as it is, it is the same as a comment, and even if you ignore the annotation you added, you will not notice it.

The Python standard does not provide an official tool for checking type annotations (as of 3.5). Therefore, you need to use an external tool for checking.

One type of this tool is mypy.

mypy

It was developed by Jukka, who is also the author of typehinting of Python itself, and I think it can be said that it is the de facto standard library for type annotation.

For checks in an integrated development environment, PyCharm has some support. In the following, the warning is where a numerical value is passed to greeting which takes str as an argument.

PyCharm originally supports annotations in docstrings and comments and will take over the standard in the future Can be expected to be added.

Visual Studio's Python Development Environment (PTVS) is said to be okay because it originally has strong type inference, but it seems that the inclusion of notation is being discussed.

Use type hints in Python 3.x to infer variable types if possible #82

install mypy

Here, we will introduce the installation procedure for checking using mypy.

You can install mypy from pip.

pip install mypy

~~ * Note that pip install mypy will include a completely different library ~~ From mypy 0.470, the library name has been changed from mypy-lang to mypy

Of course, it is premised on Python 3 (it is ok even if it is not 3.5), but [Python 2 will also be supported](http://mypy.readthedocs.org/en/latest/faq.html#all-of-my-code -is-still-in-python-2-what-are-my-options). As of November 2, 2015, if you want to use mypy with Python3.5, you need to install it from master. This is because the support for ʻUndefined`, which was removed in Python 3.5, has not yet been reflected in pypi (issue 639. ).

To pip install from GitHub master, do the following:

pip install git+https://github.com/JukkaL/mypy.git@master

Use of mypy

It is as per the official Quick Start, but you can run the check with the mypy command that will be available after installation.

mypy PROGRAM

If that doesn't work, please refer to Troubleshooting. The basics are covered here.

When you actually execute it, the execution result will be as follows.

>>> mypy examples/basic.py
examples\basic.py:5: error: Argument 1 to "greeting" has incompatible type "int"; expected "str"

You can also specify the module with the -m option.

>>> mypy -m examples
examples\__init__.py:1: note: In module imported here:
examples\basic.py:5: error: Argument 1 to "greeting" has incompatible type "int"; expected "str"

Here, ʻexamples / __ init __. Py` is as follows.

import examples.basic

As the check says "ʻIn module imported here: ", when targeting packages, it seems that anything that does not follow from __ init __. Py` is not checked. So be careful if you want to check all the files in the module.

As for existing libraries, it is not possible to annotate the source code of the contents and go around, so it is also possible to cut out the type definition to the outside and define it. A file that describes only this type definition is called a stub file, and its extension is pyi (TypeScript d.ts image).

If you want to read the pyi file prepared by mypy, set MYPYPATH.

export MYPYPATH=~/work/myproject/stubs

If you use the --use-python-path option, the ones in PYTHONPATH will be referenced.

Currently, there is no such thing as a type definition management tool like tsd in TypeScript, but [python / typeshed](https://github. It seems that aggregation is done to com / python / typeshed) (although there are not many yet).

For more details, please refer to the Official Document.

Introduction policy

The following points can be expected as the effects of introducing TypeHints.

Shallow unit tests: Serve as a test to prevent run-time errors due to misuse of functions
Applicable to places where type strictness is required, such as around the database
Clarify the type and check for things with complicated specifications such as dates (whether it is UTC, notation, etc.)
Type explicit when using dynamic call etc.
Clarification of specification: Clarify what is allowed as an argument and what value may be returned
Suggestions about the need for None support by using ʻOptional`
In cases where it is unclear whether the caller will pre-process or handle it (ex. Whether to cast before calling, if there is one element, make it an array of one element, etc.), Clarify the division by clarifying the specifications
Clarify the specifications of functions used as Callback in advance (delegate)

And I think that the applicable targets are roughly as follows.

Complete target: For all processes
Partial target: Targets the parts set based on certain criteria, such as processing around the DB and specific modules.

The following patterns can be considered as the method of introducing annotations for the specified target.

Full introduction: Annotate all applicable targets
Sequential introduction: Determine whether it is better to add it by designing in advance or code review.
Free introduction: Like comments, attach when the developer feels it is necessary

When introducing it, I think it is necessary to first clarify what kind of benefits you expect. If this is not clear, the criteria for application and implementation will be ambiguous.

This area is undergoing trial and error, including whether the expected effect can be obtained in the first place, so I would like to add it when know-how is accumulated again.