I want to display the progress in Python!

Overview

When you execute a process that takes a long time, if there is no response, do you feel anxious about "how far the process is progressing" or "is it working?" There is, right? Yes, there is (three-row conjugation). So I will make a note of how to display the progress of loop processing.

↓ ↓ ↓ By the way, it looks like this ↓ ↓ ↓ progress_DB.gif If you find it useful, I would be grateful if you could comment on LGTM stock.

table of contents

-[Use package tqdm](#Use package tqdm) -[Try to make it yourself](# Try to make it yourself) -[Escape Sequence](# Escape Sequence) -[Try to make a progress bar](# Try to make a progress bar) -[Development: ANSI escape code](#Development ansi escape code)

Use of package tqdm

First of all, I would like to introduce tqdm, the royal road of progress display (I think it is selfish). It's very easy to use, just import and embed it in your loop like this:

tqdm_test.py


%!pip install tqdm
import tqdm
import numpy as np


for i in tqdm.tqdm(range(int(1e7))):
    np.pi*np.pi

Now you should see something like this: tqdm_test.gif It's convenient ~ Since you are passing an iterable (loopable) object to tqdm.tqdm (), you can also pass lists, dictionaries, strings, and so on. However, as a caveat, if there is a standard output print function etc. in the loop processing, it will be terrible.

tqdm_test.py


%!pip install tqdm
import tqdm
import numpy as np


for i in tqdm.tqdm(range(int(1e7))):
    np.pi*np.pi
    if i % 1e6 == 0:
        print(i)

tqdm_test_fail.png By the same theory, it would be terrible to show progress with the tqdm.tqdm function for nesting loops.

tqdm_test.py


import tqdm
for t in tqdm.tqdm(range(10)):
    for i in tqdm.tqdm(range(int(1e6))):
        np.pi*np.pi

tqdm_test_nest.gif This kind of thing is very rarely inconvenient, isn't it? Do you all have such an experience? There is, right? Yes, there is (three-row conjugation). So let's do something about it ourselves.

Try to make it yourself

At least in Python, it seems that the standard output string leaves control of the program at the time of line break **. Conversely, it means that it can be operated programmatically even after standard output as long as there is no line break **.

progress.py


for i in range(int(1e7)):
    if i % 1e4 == 0:
        print("\r", i, end="")
    np.pi*np.pi

test_print_r.gif These \ r andend = ""allow the program to continue to take control after standard output. end =" " is an option to specify the character string to be added to the end of the character string output by the print function, and \ n (line feed) is specified by default. Therefore, if you change it to add an empty string by setting end =" " (although it is strange to say), line breaks will not occur, so you can play with the standard output programmatically. As it appeared here, \ r and \ n starting with a backslash are called ** escape sequences **, and are a group of strings that enable special control that you want to include in the string. I will.

Escape sequence

As mentioned above, an escape sequence is a group of character strings used when you want to give special control to a character string. The typical ones are as shown in the table below.

Escape sequence effect
\b Backspace
\n new line
\t tab
\r Back to top of line

In terms of processing, the backslash indicates the beginning of the escape sequence description, and it is converted to the control statement or character code corresponding to the character string that follows it. Therefore, it is also used when outputting the character string used in the control statement of the character string itself such as double quotation " .

test_esc.py


print("\"")
print("\'")
print("\\")

It's possible to use this feature to read a string in Unicode, but that's probably not the case.

test_esc.py


# Hello world!
print("\x48\x65\x6c\x6c\x6f\x20\x77\x6f\x72\x6c\x64\x21")

Try to make a progress bar

Let's create our own progress bar using the escape sequence \ r.

progress.py


import time
for i in range(50):
    print("\r[" + "#"*i + "]", end="")
    time.sleep(0.1)

progress_first.gif For the time being, we have created something that can be called a progress bar! By the way, I don't know why, but the escape sequence doesn't work when I use the format function. It doesn't taste good as it is, so let's try more.

progress.py


import time
epoch = 50
for i in range(epoch):
    bar = "="*i + (">" if i < epoch-1 else "=") + " "*(epoch-i-1)
    print("\r[" + bar + "]", "{}/{}".format(i+1, epoch), end="")
    time.sleep(0.1)

As an explanation, first write the main character string of the progress bar in the variable bar. In the code, spaces are filled in to fix the length of the progress bar, and the beginning is an arrow.

Besides, you can do anything because you only have to define the nesting that you couldn't do with tqdm.

progress.py


import time


t_epoch = 10
i_epoch = 50
lap_time = -1
start_time = time.time()
for t in range(t_epoch):
    t_per = int((t+1)/t_epoch*i_epoch)
    for i in range(i_epoch):
        i_per = int((i+1)/i_epoch*i_epoch)
        if i_per <= t_per:
            bar = ("progress:[" + "X"*i_per
                                + "\\"*(t_per-i_per)
                                + " "*(i_epoch-t_per) + "]")
        else:
            bar = ("progress:[" + "X"*t_per
                                + "/"*(i_per-t_per)
                                + " "*(i_epoch-i_per) + "]")
        time.sleep(1e-2)
        elapsed_time = time.time() - start_time
        print("\r" + bar, "{}s/{}s".format(
                int(elapsed_time),
                int(lap_time*t_epoch) if lap_time > 0 
           else int(elapsed_time*(i_epoch/(i+1))*t_epoch)),
              end="")
    lap_time = (time.time() - start_time)/(t+1)

progress_second.gif It suddenly became complicated ... I will explain. t_per and i_per calculate and hold the number of characters required to display the progress in the t loop and i loop as a character string. bar is the progress bar itself

--The t loop uses the string\to indicate progress. --The i loop uses the string/to indicate progress. --The overlapping part is the character string X to indicate the progress.

It is programmed to be like this. The backslash is the start character of the escape sequence, so you need to use " \\ ". lap_time holds the time taken for one loop of i. I take the average to get a more accurate value. elapsed_time is the elapsed time to the present.

progress.py


                int(lap_time*t_epoch) if lap_time > 0 
           else int(elapsed_time*(i_epoch/(i+1))*t_epoch))

However, if lap_time is calculated, it is displayed, and if it is not (that is, when the loop of i is entered for the first time), the lap time is estimated from the elapsed time and displayed.

This is just an example, so please think about it.

Evolution: ANSI escape code

So far, I said that "the standard output cannot be overwritten if there is a line break", but in fact, it can be operated even if there is a line break. The method is ** ANSI escape code **.

progress.py


import time
epoch = 25

print(" "*(epoch-10) + "彡 ノ ノ ノ Hami ⌒ Mi")
print(" "*(epoch-11) + " (´ ・ ω ・ `)ω ・ `)now! Shoot with Ora!")
print(" "*(epoch-13) + "⊂ ∩ ∩ tsu)")
print(" "*(epoch-10) + "/   〈   〈")
print(" "*(epoch-11) + " ( /⌒ ` J ⌒'")
print("\n\n")
print(" "*(epoch-1)  + "No")
print(" "*(epoch-4)  + "彡 ノ")
print(" "*(epoch-6)  + "No")
print(" "*(epoch-8)  + "Nono Minono")
print()
print(" "*(epoch-11) + "(´;ω;`)ω^`)Shit ah ah ah! !! !!")
print(" "*(epoch-13) + "⊂ ∩ ∩ tsu)")
print(" "*(epoch-10) + "/   〈   〈")
print(" "*(epoch-11) + " ( /⌒ ` J ⌒'")
print("\033[6A")
for i in range(epoch):
    bar = "弌"*i + "⊃" + " "*(epoch-i-1)
    print("\to r" + bar + "]", "{}/{}".format(i+1, epoch), end="")
    time.sleep(0.1)
print()
print("\033[5B")

progress_DB.gif Please pass through if the way to hide personal information is complicated lol Do you know It is a parody AA of the famous scene of Dragon Ball, the game against Raditz. (I found something, so I used it) The code to display the AA part is pretty much what you see. However, note that all AAs have been output in advance. And

progress.py


print("\033[6A")

Pre-opened by

progress.py


print(" "*(epoch-8)  + "Nono Minono")
print()
print(" "*(epoch-11) + "(´;ω;`)ω^`)Shit ah ah ah! !! !!")

I moved to the empty line part of and displayed the Manuki Kosatsu method in the same way as before. Finally

progress.py


print()
print("\033[5B")

With, the Manuki Kosatsu method is broken and moved to the last line of standard output.

Now, the mysterious code is coming out, this is the ** ANSI escape code **. In Python, there are ANSI escape codes in the following table.

ANSI escape code effect
\033[nA Move cursor up n lines
\033[nB Move cursor down n lines
\033[nC Move the cursor to the right n
\033[nD Move the cursor to the left n
\033[nE Move the cursor down n lines and then move to the beginning of that line
\033[nF Moves the cursor up n lines and then moves to the beginning of that line
\033[nG Move the cursor to the nth position counting from the left end
\033[n;mH Move the cursor to the nth line counting from the top of the console and to the m position counting from the left end
\033[nJ n=When it is 0, the character string after the cursor(Includes subsequent lines)Delete all, n=When it is 1, the character string before the cursor(Includes previous line)Delete, n=When it is 2, the whole character string(All output)Delete
\033[nK ~~n=When it is 0, the character string after the cursor is deleted, n=When it is 1, the character string before the cursor is deleted, n=If it is 2, delete the entire line~~It may not work in Python.
\033[nS Scroll n lines console next
\033[nT Scroll n lines forward on the console
\033[n;mf Move the cursor as with H
\033[nm SGR:Select Graphic Rendition command. Perform graphic control. Detail isHere

If you have any questions, let's make only the first four available. By using this ANSI code, you can operate with a high degree of freedom as before.

By the way, this ANSI code seems to be usable only when using a console such as a terminal, so let's put up with the single-line progress bar at \ r in jupyter notebook etc.

in conclusion

So I made a memorandum about the unexpectedly complicated standard output. Well, it seems unlikely that people who need to make their own progress bar ... Well, I'm sure it's good that you can play various things by using these functions.

reference

-Escape Sequence -Unicode list -Aa of Manuki Kotobuki was completed -2ch cute AA / emoticon summary

Recommended Posts

I want to display the progress in Python!
I want to display the progress bar
I want to write in Python! (3) Utilize the mock
I want to use the R dataset in python
I want to do Dunnett's test in Python
I want to create a window in Python
I want to merge nested dicts in Python
I want to batch convert the result of "string" .split () in Python
I want to explain the abstract class (ABCmeta) of Python in detail.
I want to write in Python! (1) Code format check
I want to embed a variable in a Python string
I want to easily implement a timeout in python
I want to write in Python! (2) Let's write a test
Even in JavaScript, I want to see Python `range ()`!
I want to randomly sample a file in Python
I want to inherit to the back with python dataclass
I want to work with a robot in python.
I want to do something in Python when I finish
I want to manipulate strings in Kotlin like Python!
Python OpenCV tried to display the image in text.
[Python] I want to know the variables in the function when an error occurs!
I want to use Python in the environment of pyenv + pipenv on Windows 10
In the python command python points to python3.8
I wrote the queue in Python
I want to get the file name, line number, and function name in Python 3.4
I wrote the stack in Python
I want to initialize if the value is empty (python)
maya Python I want to fix the baked animation again.
I want to do something like sort uniq in Python
[Python] I want to use the -h option with argparse
I tried to implement the mail sending function in Python
I want to know the features of Python and pip
I want to make the Dictionary type in the List unique
I want to align the significant figures in the Numpy array
I want to be able to run Python in VS Code
I want to make input () a nice complement in python
I didn't want to write the AWS key in the program
I just want to find the 95% confidence interval for the difference in population ratios in Python
I want to replace the variables in the python template file and mass-produce it in another file.
I want to pin Spyder to the taskbar
I tried to implement PLSA in Python
I want to output to the console coolly
I want to print in a comprehension
I want to handle the rhyme part1
I want to handle the rhyme part3
I tried to implement PLSA in Python 2
Display Python 3 in the browser with MAMP
I want to use jar from python
I want to build a Python environment
I want to analyze logs with Python
How to display multiplication table in python
I want to play with aws with python
I tried to implement ADALINE in Python
I wanted to solve ABC159 in Python
I tried to implement PPO in Python
I want to embed Matplotlib in PySimpleGUI
How to display Hello world in python
I want to handle the rhyme part2
I want to handle the rhyme part5
I want to handle the rhyme part4
[Linux] I want to know the date when the user logged in