I want to do something in Python when I finish

I want to do something in Python when I finish

I make a directory on the way, but I don't want it to remain at the end. I put temporary data in the database, but I don't want it to remain at the end. Please let me know when the program is finished, whether it is normal or dead.

This is a way to achieve this.

This time, I will aim for the following things.

  1. Of course I want you to do it if it is normal
  2. I want you to execute even if an Exception occurs
  3. I want you to execute even if you stop with Ctrl-C
  4. I want you to execute even if you stop with kill
  5. Don't stop during the Cleanup process
  6. Give up kill -9 and Segmentaion Fault

I will try several methods. For those who are in a hurry, I will write the conclusion first.

Conclusion first

import sys
import time
import signal


def setup():
    print("!!!Set up!!!")


def cleanup():
    print("!!!Clean up!!!")
    #Cleanup processing various
    time.sleep(10)
    print("!!!Clean up Done!!!")


def sig_handler(signum, frame) -> None:
    sys.exit(1)


def main():
    setup()
    signal.signal(signal.SIGTERM, sig_handler)
    try:
        #Various processing
        time.sleep(60)

    finally:
        signal.signal(signal.SIGTERM, signal.SIG_IGN)
        signal.signal(signal.SIGINT, signal.SIG_IGN)
        cleanup()
        signal.signal(signal.SIGTERM, signal.SIG_DFL)
        signal.signal(signal.SIGINT, signal.SIG_DFL)


if __name__ == "__main__":
    sys.exit(main())

Commentary

Preparation

Prepare the setup () / clean () function and the main part.

def setup():
    print("!!!Set up!!!")


def cleanup():
    print("!!!Clean up!!!")

def main():
    pass

if __name__ == "__main__":
    sys.exit(main())

use try --finally

The first thing that comes to mind is to use try --finally.

def main():
    setup()
    try:
        print("Do some jobs")

    finally:
        cleanup()

Let's run it.

!!!set up!!!
do some jobs
!!!Clean up!!!

It has been cleaned up. But what if an error occurs along the way?

def main():
    setup()
    try:
        print(1 / 0)

    finally:
        cleanup()

I will try it.

!!!set up!!!
!!!Clean up!!!
Traceback (most recent call last):
  File "./try-finally.py", line 26, in <module>
    sys.exit(main())
  File "./try-finally.py", line 19, in main
    print(1 / 0)
ZeroDivisionError: division by zero

This has also been cleaned up.

But what if you stop with Ctrl-C?

def main():
    setup()
    try:
        time.sleep(60)

    finally:
        cleanup()

Run it and press Ctrl-C in the middle to stop it.

!!!Set up!!!
^C!!!Clean up!!!
Traceback (most recent call last):
  File "./try-finally.py", line 27, in <module>
    sys.exit(main())
  File "./try-finally.py", line 20, in main
    time.sleep(60)
KeyboardInterrupt

Apparently it worked.

But what if you kill it with kill?

!!!Set up!!!
Terminated

This should not be done. This will leave garbage files behind.

Conclusion

try --finally

  1. Of course I want you to do it if it is normal ⇨ ○
  2. I want you to execute even if an Exception occurs ⇨ ○
  3. I want you to execute even if you stop with Ctrl-C ⇨ ○
  4. I want you to execute it even if you stop it with kill ⇨ ×

use atexit

Python has an atexit that runs it on exit. Let's try this.

def main():
    setup()
    atexit.register(cleanup)

    print("Do some jobs")

Let's run it.

!!!Set up!!!
Do some jobs
!!!Clean up!!!

It went well. Next, let's get an error.

def main():
    setup()
    atexit.register(cleanup)

    print(1 / 0)

I will try it.

!!!Set up!!!
Traceback (most recent call last):
  File "./try-finally.py", line 25, in <module>
    sys.exit(main())
  File "./try-finally.py", line 21, in main
    print(1 / 0)
ZeroDivisionError: division by zero
!!!Clean up!!!

This also worked. Compared to the case of try --finally, the clean up process is later.

So what if you stop with Ctrl-C?

def main():
    setup()
    atexit.register(cleanup)

    time.sleep(60)

Run it and try to stop it with Ctrl-C in the middle.

!!!Set up!!!
^CTraceback (most recent call last):
  File "./try-finally.py", line 25, in <module>
    sys.exit(main())
  File "./try-finally.py", line 21, in main
    time.sleep(60)
KeyboardInterrupt
!!!Clean up!!!

feel well.

Then, finally, let's stop with kill.

!!!Set up!!!
Terminated

Sorry. Like try-finally, it doesn't seem to work either.

Conclusion

in atexit

  1. Of course I want you to do it if it is normal ⇨ ○
  2. I want you to execute even if an Exception occurs ⇨ ○
  3. I want you to execute even if you stop with Ctrl-C ⇨ ○
  4. I want you to execute it even if you stop it with kill ⇨ ×

use signal

The kill command sends a signal called SIGTERM to the process. Ctrl-C also sends a signal called SIGINT to the process. Let's try trapping these signals.

def sig_handler(signum, frame) -> None:
    cleanup()
    sys.exit(1)


def main():
    setup()
    signal.signal(signal.SIGTERM, sig_handler)
    signal.signal(signal.SIGINT, sig_handler)

    print("Do some jobs")

I will try it.

!!!Set up!!!
Do some jobs

Clean up will not be performed. That's because the signal hasn't flown from the outside. It is natural. Since Exception is the same, confirmation is omitted.

Now let's see what happens with Ctrl-C.

def main():
    setup()
    signal.signal(signal.SIGTERM, sig_handler)
    signal.signal(signal.SIGINT, sig_handler)

    time.sleep(60)

Run it and try to stop it with Ctrl-C in the middle.

!!!Set up!!!
^C!!!Clean up!!!

It's perfect. The KeyboardInterrupt Exception has disappeared, but it shouldn't usually be a problem. If absolutely necessary, you can raise (KeyboardInterrupt ()) in sig_handler.

Then, it is a kill to be worried about, but what will happen?

!!!Set up!!!
!!!Clean up!!!

Great! It was cleaned up with chitin.

Conclusion

  1. Of course I want you to do it if it is normal ⇨ ×
  2. I want you to execute even if an Exception occurs ⇨ ×
  3. I want you to execute even if you stop with Ctrl-C ⇨ ○
  4. I want you to execute it even if you stop it with kill ⇨ ○

So, try --finally or atexit and signal should be combined.

use try --finally in combination with signal

This time, assuming that the scope is narrow, I will try to combine try --finally and signal.

def sig_handler(signum, frame) -> None:
    cleanup()
    sys.exit(1)


def main():
    setup()
    signal.signal(signal.SIGTERM, sig_handler)
    try:
        print("do some jobs")

    finally:
        signal.signal(signal.SIGTERM, signal.SIG_DFL)
        cleanup()

It became like this. Ctrl-C also works with try --finally, so I tried to trap only SIGTERM. After exiting the try, cleanup should not be needed, so finally set SIGTERM back to the default.

Let's run it.

!!!Set up!!!
do some jobs
!!!Clean up!!!

As expected.

Next, let's get an error.

def main():
    setup()
    signal.signal(signal.SIGTERM, sig_handler)
    try:
        print(1 / 0)

    finally:
        signal.signal(signal.SIGTERM, signal.SIG_DFL)
        cleanup()


if __name__ == "__main__":
    sys.exit(main())

I will try it.

!!!Set up!!!
!!!Clean up!!!
Traceback (most recent call last):
  File "./try-finally.py", line 33, in <module>
    sys.exit(main())
  File "./try-finally.py", line 26, in main
    print(1 / 0)
ZeroDivisionError: division by zero

It has been cleaned up.

Next, check Ctrl-C.

def main():
    setup()
    signal.signal(signal.SIGTERM, sig_handler)
    try:
        time.sleep(60)

    finally:
        signal.signal(signal.SIGTERM, signal.SIG_DFL)
        cleanup()

Run it and try pressing Ctrl-C in the middle.

!!!Set up!!!
^CClean up!!
Traceback (most recent call last):
  File "./try-finally.py", line 33, in <module>
    sys.exit(main())
  File "./try-finally.py", line 26, in main
    time.sleep(60)
KeyboardInterrupt

This has also been cleaned up.

Now, what about kill?

!!!Set up!!!
!!!Clean up!!!
!!!Clean up!!!

eh! It has been cleaned up twice.

By trapping SIGTERM, it seems that Python did not kill and executed finally. So, let's change sig_handler as follows.

def sig_handler(signum, frame) -> None:
    sys.exit(1)

I stopped calling cleanup in the handler and just finished it.

Run it and try killing it.

!!!Set up!!!
!!!Clean up!!!

It's finally as expected.

Conclusion

  1. Of course I want you to do it if it is normal ⇨ ○
  2. I want you to execute even if an Exception occurs ⇨ ○
  3. I want you to execute even if you stop with Ctrl-C ⇨ ○
  4. I want you to execute it even if you stop it with kill ⇨ ○

Do not stop during the Cleanup process

The previous method is sufficient, but it is human nature to hit Ctrl-C repeatedly if you do not stop immediately after pressing Ctrl-C. You may also kill it many times.

I made the Cleanup process work, but that's a mess. So, I will not stop it during Cleanup.

def main():
    setup()
    signal.signal(signal.SIGTERM, sig_handler)
    try:
        time.sleep(60)

    finally:
        signal.signal(signal.SIGTERM, signal.SIG_IGN)
        signal.signal(signal.SIGINT, signal.SIG_IGN)
        cleanup()
        signal.signal(signal.SIGTERM, signal.SIG_DFL)
        signal.signal(signal.SIGINT, signal.SIG_DFL)

Ignore Ctrl-C and kill signals before cleanup (). After cleanup () is done, restore the default. This is perfect.

Conclusion

  1. Of course I want you to do it if it is normal ⇨ ○
  2. I want you to execute even if an Exception occurs ⇨ ○
  3. I want you to execute even if you stop with Ctrl-C ⇨ ○
  4. I want you to execute it even if you stop it with kill ⇨ ○
  5. Do not stop in the middle of the termination process ⇨ ○

bonus

Try --finally / atexit properly

Looking at the results this time, it seems that the only difference between try-catch and atexit is the difference in scope and timing. If setup runs in the middle of a program, use try --finally at that scope. If setup is the first thing you run in your program, use atexit. If you want it to run anytime at the end, regardless of setup, use atexit. Does that mean?

Also, there is a way to use with, so if you are interested, please try it.

Recommended Posts

I want to do something in Python when I finish
I want to do something like sort uniq in Python
I wanted to do something like an Elixir pipe in Python
I want to create a window in Python
I want to merge nested dicts in Python
I want to display the progress in Python!
I want to do a monkey patch only partially safely in Python
I want to write in Python! (1) Code format check
I want to embed a variable in a Python string
I want to easily implement a timeout in python
I want to write in Python! (2) Let's write a test
Even in JavaScript, I want to see Python `range ()`!
I want to randomly sample a file in Python
I want to work with a robot in python.
I want to write in Python! (3) Utilize the mock
[ML Ops] I want to do multi-project with Python
I want to manipulate strings in Kotlin like Python!
[Python] I want to know the variables in the function when an error occurs!
I want to improve efficiency with Python even in an experimental system (3) I want to do something like Excel with Pandas
[Python] How to do PCA in Python
I want to do ○○ with Pandas
I want to debug with Python
What to do when "SSL: CERTIFICATE_VERIFY_FAILED _ssl.c: 1056" appears in Python
I want to make input () a nice complement in python
[Subprocess] When you want to execute another Python program in Python code
I tried to implement permutation in Python
I want to print in a comprehension
How to do R chartr () in Python
I want to solve APG4b with Python (only 4.01 and 4.04 in Chapter 4)
I tried to implement PLSA in Python 2
I want to build a Python environment
I want to run the Python GUI when starting Raspberry Pi
I want to do a full text search with elasticsearch + python
[Python] When you want to use all variables in another file
I want to analyze logs with Python
I want to play with aws with python
[Python / AWS Lambda layers] I want to reuse only module in AWS Lambda Layers
I tried to implement ADALINE in Python
I wanted to solve ABC159 in Python
I want to embed Matplotlib in PySimpleGUI
What to do when ModuleNotFoundError: No module named'XXX' occurs in Python
When I got a list of study sessions in Python, I found something I wanted to make
What to do when the value type is ambiguous in Python?
I want to do it with Python lambda Django, but I will stop
I want to convert a table converted to PDF in Python back to CSV
I want to batch convert the result of "string" .split () in Python
I want to explain the abstract class (ABCmeta) of Python in detail.
I want to color a part of an Excel string in Python
After all, what should I use to do type comparisons in Python?
I want to use MATLAB feval with python
I want to pin Datetime.now in Django tests
I want to memoize including Python keyword arguments
[Python] What I did to do Unit Test
I was able to recurse in Python: lambda
I want to email from Gmail using Python.
When I try matplotlib in Python, it says'cairo.Context'
[Python] I want to manage 7DaysToDie from Discord! 1/3
Minimal implementation to do Union Find in Python
I want to make a game with Python
I wrote "Introduction to Effect Verification" in Python
I want to store DB information in list