Your threading.Event is used incorrectly

Introduction

When I was googled for a sample using Python's threading.Event, I found one that was misused at the top. Even in the post using threading.Event on Qiita, it was a miserable situation that all 3 posts were used incorrectly, so I will explain the correct usage.

What is threading.Event?

This class is used to make a thread wait until an event occurs, and when an event is generated from another thread, the waiting thread resumes.

The two most important methods are:

There are also clear () and is_set (). See [Python documentation] threading.Event for more information.

Correct usage

First, let's see how to use it correctly.

Simple example

This is the simplest example using only wait () and set (). The thread waits until the event occurs, and the main thread generates the event 3 seconds after the thread starts.

The first half is a little long because the time and thread name are set to be output to the log, but don't worry.

from logging import (getLogger, StreamHandler, INFO, Formatter)

#Log settings
handler = StreamHandler()
handler.setLevel(INFO)
handler.setFormatter(Formatter("[%(asctime)s] [%(threadName)s] %(message)s"))
logger = getLogger()
logger.addHandler(handler)
logger.setLevel(INFO)


from threading import (Event, Thread)
import time


event = Event()


def event_example1():
    logger.info("Thread start")
    event.wait()
    logger.info("End thread")

thread = Thread(target=event_example1)
thread.start()
time.sleep(3)
logger.info("Event occurrence")
event.set()

Execution result


[2016-09-27 00:04:18,400] [Thread-6]Thread start
[2016-09-27 00:04:21,406] [MainThread]Event occurrence
[2016-09-27 00:04:21,407] [Thread-6]End thread

If you look at the time, you can see that the thread is waiting for the event to occur.

Call wait with timeout

If you specify a timeout, the thread resumes after the specified number of seconds, even if no event occurs. The return value of wait () is True when the event occurs, False otherwise.

python


event = Event()


def event_example2():
    logger.info("Thread start")
    while not event.wait(2):
        logger.info("That's right")
    logger.info("End thread")

thread = Thread(target=event_example2)
thread.start()
time.sleep(5)
logger.info("Event occurrence")
event.set()

Execution result


[2016-09-27 00:04:21,407] [Thread-7]Thread start
[2016-09-27 00:04:23,412] [Thread-7]That's right
[2016-09-27 00:04:25,419] [Thread-7]That's right
[2016-09-27 00:04:26,409] [MainThread]Event occurrence
[2016-09-27 00:04:26,409] [Thread-7]End thread

You can see that it is timing out even if the event does not occur. Also, when an event occurs, it restarts before the timeout.

Use Event repeatedly

As stated in the documentation, once you call set (), even if you call wait (), the thread will return without waiting. Therefore, if you want to use Event repeatedly, you need to call clear () to clear the event.

The following is an example using clear (). It also uses the bool flag stop separately from ʻevent` to terminate the thread.

python


event = Event()

#Event stop flag
stop = False


def event_example3():
    logger.info("Thread start")
    count = 0
    while not stop:
        event.wait()
        event.clear()
        count += 1
        logger.info(count)
    logger.info("End thread")

thread = Thread(target=event_example3)
thread.start()

time.sleep(1)
event.set()
time.sleep(1)
event.set()
time.sleep(1)
stop = True
event.set()

thread.join()

Execution result


[2016-09-27 00:04:26,410] [Thread-8]Thread start
[2016-09-27 00:04:27,415] [Thread-8] 1
[2016-09-27 00:04:28,417] [Thread-8] 2
[2016-09-27 00:04:29,421] [Thread-8] 3
[2016-09-27 00:04:29,421] [Thread-8]End thread

Wrong usage

Next, let's look at the wrong usage that came out by searching.

Mistake pattern 1 --using threading.Event as just a flag

How to terminate a process with multiple threads that loop infinitely --Qiita Operate the running thread from the outside --Qiita Multi-threaded Zundokokiyoshi using Event and Queue --Qiita

This pattern uses only set () and is_set () and uses Event only as a flag. It may be an overstatement to say that it is wrong, but there is no point in using Event unless you call wait ().

Example of using Event as just a flag


event = Event()


def bad_example1():
    logger.info("Thread start")
    while not event.is_set():
        logger.info("That's right")
        time.sleep(2)
    logger.info("End thread")

thread = Thread(target=bad_example1)
thread.start()
time.sleep(5)
logger.info("Event occurrence")
event.set()

In such cases, it is simpler to replace the time.sleep () and is_set () parts in the thread with Event.wait (), or to use bool without Event.

wait()use


event = Event()


def bad_example1():
    logger.info("Thread start")
    while not event.wait(timeout=2):
        logger.info("That's right")
    logger.info("End thread")

thread = Thread(target=bad_example1)
thread.start()
time.sleep(5)
logger.info("Event occurrence")
event.set()

Use bool flag


stop = False


def bad_example1():
    logger.info("Thread start")
    while not stop:
        logger.info("That's right")
        time.sleep(2)
    logger.info("End thread")

thread = Thread(target=bad_example1)
thread.start()
time.sleep(5)
logger.info("Event occurrence")
stop = True

Mistake pattern 2-Forgot clear ()

[Try Python threading.Event | CUBE SUGAR STORAGE](http://momijiame.tumblr.com/post/38384408139/python-%E3%81%AE-threadingevent-%E3%82%92%E8% A9% A6% E3% 81% 97% E3% 81% A6% E3% 81% BF% E3% 82% 8B)

First, take a look at the sample link above.

In the BlockingQueue class, if the queue is empty when pop () is called, it waits until an event occurs, and in push (), when a value is added to the queue, an event is generated and the waiting thread that called pop () is restarted.

The Consumer calls pop () repeatedly, and the Producer calls push () every second to add a random value to the queue.

Looking at the source, it uses wait (), and even if it is executed, a random value is displayed at 1 second intervals, so it looks like it is working as expected.

Execution result


27
88
53
148
:

However, if you put a log in the wait () part, you can see that it is behaving ridiculously.

    def pop(self):
        # wait()Loop to return the thread that exited
        while True:
            #Get the lock
            with self.lock:
                if self.queue:
                    #Returns any elements in the queue
                    return self.queue.pop()
                else:
                    print("waiting...")  # <---add to
                    #If the queue is empty, wait for another thread to add an element and notify you
                    self.event.wait()

Execution result


waiting...
waiting...
waiting...
waiting...
:

You will see a lot of waiting ... and you can see that the threat is not waiting at all with wait ().

Cause and correction

This is because clear () is not called, so even if you call wait () once you call set (), it will not wait.

In order to make this work properly, you can call clear (), but there is also a bug that it will be in a deadlock state because it is waiting () while holding the lock, so it is as follows Needed to be fixed.

Revised


    def pop(self):
        # wait()Loop to return the thread that exited
        while True:
            self.event.clear()
            #Get the lock
            with self.lock:
                if self.queue:
                    #Returns any elements in the queue
                    return self.queue.pop()
            print "waiting..."
            #If the queue is empty, wait for another thread to add an element and notify you
            self.event.wait()

Execution result


7
waiting...
169
waiting...
113
waiting...

Now it waits properly.

By the way

There is a queue in the Python standard module.

This is a multithreaded queue that allows you to:

--Wait a thread if the queue is empty when you call pop () --If the queue is full when you call push (), wait until it is free

In the BlockingQueue that appears in this sample, when pop () is called, if the queue is empty, it waits for a thread, but this has already been realized in the standard module.

I think this sample is just written as an explanation of Event. If you want to use a queue with multiple threads, use the standard module queue instead of creating it yourself.

Summary

--threading.Event is meaningless without using wait () --Use bool if you want to use flags --Don't forget to call clear () for repeated use --If you want to use a queue with multiple threads, use the standard module queue.

Recommended Posts

Your threading.Event is used incorrectly
How Python __dict__ is used
Your multi-layer perceptron is dirty
What is your "Tanimoto coefficient"?