[Python] How to write type annotations for Callable objects treated as variables and arguments

There are times when you need to treat a function or method as a variable or argument (Callable object) in Python.

I will write an article about the type annotation in such a case as a memorandum.

Environment to use

First, there is no type annotation

Suppose you have a function like the one below, which accepts a function object (func) and an arbitrary value (x) as arguments.

def any_function(func, x):
    return func(x)

Obviously, if you do this, even if you mouse over VS Code, the information such as the type will be unknown. Type check is not performed even with Pylance (Pyright). There is also a countermeasure to write the details in detail with docstring etc., but if it is left as it is, it is ambiguous and it is easy to make mistakes.

image.png

Callable type annotation

Next, add the Callable type annotation to the func argument. Callable is in the built-in typing package. In addition, type annotation is also added to the x argument.

from typing import Callable


def any_function(func: Callable, x: int) -> int:
    return func(x)

This will recognize that the func argument is some kind of function or method that you can call when you hover over VS Code.

Also, if you specify something other than a function in the func argument, it will be caught in the Pylance check.

any_function(func=10, x=20)

image.png

However, the information of the argument and the return value when the mouse is over remains Unknown.

image.png

Type annotation of arguments and return information

Type annotations such as returning the func function as an int as the first argument, float as the second argument, and float as the return value (add the flaot argument y to the sample argument as well). ..

As a way of writing, write as Callable [[type of first argument, type of second argument], type of return value]. When the argument increases or decreases, it is increased or decreased separated by commas.

def any_function(
        func: Callable[[int, float], float],
        x: int,
        y: float) -> float:
    return func(x, y)

As a result, the type information of the function argument and return value is displayed when the mouse is over, and when the func argument is called, if the argument is insufficient or the type does not match, it can be detected as an error.

image.png

Sample with insufficient arguments at func call location


def any_function(
        func: Callable[[int, float], float],
        x: int,
        y: float) -> float:
    return func(x)

image.png

Samples with mismatched types in func call


def any_function(
        func: Callable[[int, float], float],
        x: int,
        y: float) -> float:
    return func(y, x)

image.png

When passing it as an argument, it is checked whether the structure of the specified function matches.

Sample that the return value of Callable passed to the argument does not match and is caught in the check


def any_function(
        func: Callable[[int, float], float],
        x: int,
        y: float) -> float:
    return func(x, y)


def other_function(x: int, y: float) -> str:
    ...


any_function(func=other_function, x=10, y=12.5)

image.png

Even with the support so far, it has become much more robust than the case without type annotation.

However, in this case, the original argument name cannot be used when the keyword argument is specified. The argument names are not preserved and are assigned the names p0, p1, .... For example, if you write with keyword arguments as shown below, it will be caught in the Pylance check.

def any_function(
        func: Callable[[int, float], float],
        x: int,
        y: float) -> float:
    return func(x=x, y=y)

image.png

In many cases this is not a problem, but for example, if you have many arguments or many arguments of the same type, you will want to use keyword arguments.

Perform type annotation to retain argument name information

In fact, you can use the typing package Protocol to perform type annotation while preserving argument names (I recently learned).

To use it, define a class that inherits Protocol, and define the arguments, return value, and type information required for the __call__ method (this time, I named it FuncType as a sample). All you have to do is specify the class (FuncType) in the type annotation of the func argument.

from typing import Callable, Protocol


class FuncType(Protocol):

    def __call__(self, x: int, y: float) -> float:
        ...


def any_function(
        func: FuncType,
        x: int,
        y: float) -> float:
    return func(x=x, y=y)

Now, even if you use keyword arguments, you will not get an error on Pylance.

image.png

Protocol is available in Python 3.8 and above. If you want to use it with Python 3.7 or earlier, or if you want to support past Python versions with distribution libraries etc., you need to install typing-extensions library with pip etc. as a backport. If you install mypy etc., it will be installed together.

The description of import also changes to typing_extensions instead of typing module. The behavior and usage are the same for both the typing Protocol and the typing_extensions Protocol.

from typing_extensions import Protocol

Reference site / reference summary

Recommended Posts

[Python] How to write type annotations for Callable objects treated as variables and arguments
Python # How to check type and type for super beginners
How to convert Python # type for Python super beginners: str
How to write a list / dictionary type of Python3
[Introduction to Python] How to write repetitive statements using for statements
About Python variables and objects
[Python] How to play with class variables with decorator and metaclass
How to define multiple variables in a python for statement
How to learn TensorFlow for liberal arts and Python beginners
How to convert Python # type for Python super beginners: int, float
How to write a metaclass that supports both python2 and python3
How to write type hints for variables that are assigned multiple times in one line
How to write a Python class
Offline real-time how to write E11 ruby and python implementation example
How to use Serverless Framework & Python environment variables and manage stages
An introduction to type annotations and Pyright for more robust Python code with rich input completion
Difference in how to write if statement between ruby ​​and python
[Python] How to create a dictionary type list, add / change / delete elements, and extract with a for statement
How to set proxy, redirect and SSL authentication for Python Requests module
How to write the correct shebang in Perl, Python and Ruby scripts
How to install MeCab (v0.996) and libraries for Python without administrator privileges
How to package and distribute Python scripts
How to access environment variables in Python
How to dynamically define variables in Python
How to install and use pandas_datareader [Python]
How to write Python document comments (Docstrings)
Type annotations for Python2 in stub files!
[Python] Organizing how to use for statements
python: How to use locals () and globals ()
[Python / Tkinter] How to pass arguments to command
How to use "deque" for Python data
[Python] How to calculate MAE and RMSE
How to use Python zip and enumerate
Compress python data and write to sqlite
How to use is and == in Python
How to write Ruby to_s in Python
How to write pydoc and multi-line comments
OpenGoddard How to use 2-python library for nonlinear optimal control and orbit generation
How to write environment variables that you don't want to put on [GitHub] Python
Tips for those who are wondering how to use is and == in Python
How to use OpenGoddard 3-python library for nonlinear optimal control and orbit generation
How to use OpenGoddard 4-python library for nonlinear optimal control and orbit generation
How to use Service Account OAuth and API with Google API Client for python
How to manage arguments when implementing a Python script as a command line tool
How to use OpenGoddard 1-python library for nonlinear optimal control and orbit generation
How to receive command line arguments in Python
[Python] How to use two types of type ()
[Introduction to Python3 Day 12] Chapter 6 Objects and Classes (6.3-6.15)
Let's understand how to pass arguments (Python version)
How to create explanatory variables and objective functions
[Python] How to read data from CIFAR-10 and CIFAR-100
How to install Python for pharmaceutical company researchers
How to convert SVG to PDF and PNG [Python]
[Introduction to Python3 Day 11] Chapter 6 Objects and Classes (6.1-6.2)
How to write a ShellScript Bash for statement
How to get dictionary type elements of Python 2.7
[Python] How to use hash function and tuple.
[Python] How to sort instances by instance variables
How to output "Ketsumaimo" as standard output in Python
How to write async and await in Vue.js
How to make Python faster for beginners [numpy]