About this article

I'd like to say that I can't really talk about "what is asynchronous programming?" This article describes what we have learned about one of the elements of asynchronous programming: collaborative multitasking, Python's ʻasync`, and event loops.

Collaborative multitasking and asynchronous I / O

Collaborative multitasking is a central element of asynchronous programming and is like an idea.

Asynchronous programming here does not perform context switching due to the scheduling function of the system, but switches inside the program.

When a computer multitasks, it does not use the context switch of the OS. When each process enters the wait state, the process itself voluntarily releases control. Gives control to many other processes running at the same time. (Comparable to the meaning of ** cooperative **) Control here refers to suspending and resuming a process, and allocating and releasing resources.

It seems that all processes need to be coordinated in order to multitask smoothly.

For application-level collaborative multitasking

Where do you do collaborative multitasking? Rather than coordinating multiple processes or threads ** All multitasking is executed in one process or thread. ** **

Who controls the task? Control of multiple tasks is limited to one function, which manages the coordination of tasks.

Key issues in collaborative multitasking

In collaborative multitasking, the most important issue is the timing of releasing control **. The timing of release is similar to the behavior of threads. Where? ⇒ Many asynchronous applications give control to the event loop or scheduler at the time of I / O instruction. Releases control when waiting for I / O processing.

The event loop will be described later.

What's the difference? ⇒ ** Thread ** is due to a system-level thread, ** OS can interrupt the running thread at any time ** and pass control to other threads. In ** asynchronous programming **, ** tasks cannot be interrupted by an event loop. ** (Non-preemptive multitasking)

What you have to keep (probably) minimal about asynchronous programming

Python runs on the OS, competing for other processes and resources. In other words, the OS controls all processes. In the case of an asynchronous application, processing is interrupted by a scheduler interrupt, but when control is returned, it resumes from the interrupted location. This is not always the case with multithreading and multiprocessing. Also, For ** multi-processes and threads **, the task to be restarted is determined by the OS scheduler **. In ** asynchronous programming **, the application ** decides which task to restart.

ʻAsync `and ʻawait` in Python

Reserved words ʻasync `and ʻawait`

ʻAsync is used before the def statement to define a new coroutine (parallel task). Execution of coroutine functions is interrupted and resumed depending on the defined situation. Even if the function defined by ʻasync is called, the function is not executed on the spot and a ** coroutine object ** is returned. The following is an implementation example.

>>> async def asyc_hello():
...   print("Hello")
...
>>> asyc_hello()
<coroutine object asyc_hello at 0x000001D3D021C748>

You can see that ʻasyc_hello ()returns a coroutine object instead of the standard output value ofprint ("Hello ")`. So what is a coroutine object? How should I handle it? I will explain this area roughly. You have to create something to execute a coroutine object. It's an ** event loop **. Below is the code that created the event loop and executed the coroutine object.

>>> import asyncio
>>> async def asyncio_hello():
...     print("Hello")
...
>>> loop = asyncio.get_event_loop()
>>> loop.run_until_complete(asyncio_hello())
Hello
>>> loop.close()

Create an event loop with ʻasyncio.get_event_loop () Execute the coroutine object byrun_until_complete (asyncio_hello ())`. Next, I will explain the basics of what the event loop is.

About the event loop

This article does not explain terms related to event loops. Instead of explaining event queues, event dispatchers, event handlers, callback functions, etc., I focused on what the event loop mechanism is doing and expressed what I learned in the figure. Below is the figure. For convenience, we changed the expression of event to request. (For convenience, it's easier to get an image of that personally) イベントループ完成図1.png

About coroutine objects

Let's return to the handling of coroutine objects. Coroutine objects will be executed inside the event loop. Also, it is queued in the event loop and the coroutine object does nothing until its turn comes. The following source code is the code that prepares only one simple coroutine and executes the event loop.

`asyncprint.py`


import asyncio

async def print_numeber(number):
    print(number)

if __name__ == "__main__":
    loop = asyncio.get_event_loop()

    loop.run_until_complete(
        asyncio.wait([
            print_numeber(number)
            for number in range(10)
        ])
    )
    loop.close()

$ python src/asyncpritn.py
2
4
8
6
9
5
1
3
7
0

This is the basic flow of the above code.

Create an event loop: ʻasyncio.get_event_loop () `
Create or add a task: ʻasyncio.get_event_loop (). create_task () ʻorʻasyncio.wait ()`
Run the event loop: ʻasyncio.get_event_loop (). run_until_complete () `
Explicitly close the event loop: ʻasyncio.get_event_loop (). close () `

Next, it is an example when the reserved word ʻawait is added to ʻasyncio.wait ().

`corowait.py`


import time 
import random
import asyncio

async def waiter(name):
    for _ in range(4):
        time_to_sleep = random.randint(1,3)/4
        time.sleep(time_to_sleep)
        print(
            "{}Is{}Waited for a second"
            "".format(name, time_to_sleep)
        )

async def main():
    await asyncio.wait([waiter("foo"), waiter("bar")])

if __name__ == "__main__":
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())
    loop.close()

$time python corowait.py
foo is 0.Waited 5 seconds
foo is 0.Waited 25 seconds
foo is 0.Waited 75 seconds
foo is 0.Waited 25 seconds
bar is 0.Waited 25 seconds
bar is 0.Waited 75 seconds
bar is 0.Waited 75 seconds
bar is 0.Waited 25 seconds

real    0m4.416s
user    0m0.130s                                                                                      sys     0m0.013s

ʻAwaitwaits for the coroutine to return execution, then releases control and passes it to the event loop until it finishes executing. Here, the processing is blocked by thetime.sleep () function. Therefore, it becomes a synchronous process and the processes are executed in order. In Python, there is a ʻasycio.sleep () function to change blocking processing to non-blocking processing and asynchronous processing. By using this, it is possible to execute asynchronous processing. Below is a sample code.

`cowait_improved.py`


import time 
import random
import asyncio

async def waiter(name):
    for _ in range(4):
        time_to_sleep = random.randint(1,3)/4
        await asyncio.sleep(time_to_sleep)
        print(
            "{}Is{}Waited for a second"
            "".format(name, time_to_sleep)
        )

async def main():
    await asyncio.wait([waiter("foo"), waiter("bar")])

if __name__ == "__main__":
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())
    loop.close()

$time python corowait_improved.py
foo is 0.Waited 25 seconds
bar is 0.Waited 75 seconds
foo is 0.Waited 75 seconds
bar is 0.Waited 75 seconds
foo is 0.Waited 75 seconds
bar is 0.Waited 25 seconds
foo is 0.Waited 25 seconds
bar is 0.Waited 25 seconds

real    0m2.442s
user    0m0.161s                                                                                      sys     0m0.017s

The functions foo and bar are executed alternately, and the processing speed is improved. It means that the coroutine has released control cooperatively.

References

https://www.atmarkit.co.jp/ait/articles/1103/23/news101_2.html Introduction to Python3 Expert Python Programming Revised 2nd Edition

[Python] Learn about asynchronous programming and event loops