Gradually display the output of the command executed by subprocess.Popen

I often used the communicate method when executing external commands using the subprocess module.

`python:subprocess.Popen.communicate()`


import subprocess

(stdoutdata, stderrdata) = subprocess.Popen(['some', 'command'], stdout=subprocess.PIPE).communicate()
print(stdoutdata)

However, if you want to output the result asynchronously such as a build script, it cannot be realized by the communicate method. As shown in Library Reference, the communicate method waits until the process ends, and then prints the standard output data. Because I will return it.

If you want to fulfill this requirement, you need to use the io module to create a stream that corresponds to the output of the child process.

(ref) http://docs.python.jp/2/library/subprocess.html#subprocess.Popen.stdout (ref) https://gist.github.com/mattbornski/3299031

`Stream corresponding to the output of the child process`


def exec_process(commands):
    process = subprocess.Popen(commands, stdout=subprocess.PIPE, bufsize=-1)
    with io.open(process.stdout.fileno(), closefd=False) as stream:
        [print(line.rstrip('\n')) for line in stream]

    #Wait until the end of the process and judge the result
    process.wait()
    if process.returncode != 0:
        print('Build process aborts.')
        sys.exit(1)

Below, a memo of some points I was addicted to

When creating a stream with ʻio.open, you need to give the file path or file descriptor (which can be obtained with (file object) .fileno ()). I don't know the pathname of subprocess.Popen.stdout, so I need to give it a file descriptor. Note that you are not giving the file object (= subprocess.Popen.stdout`) as is.

(ref) http://docs.python.jp/2/library/io.html#io.open

`python`


# process.io stdout.If given directly to open, it will end with a runtime exception.
def exec_process(commands):
    process = subprocess.Popen(commands, stdout=subprocess.PIPE, bufsize=-1)
    with io.open(process.stdout, closefd=False) as stream:
        [print(line.rstrip('\n')) for line in stream]

When creating a stream with ʻio.open, you need to specify closefd = False. The default is closefd = True`, and in that state the following error occurs.

close failed in file object destructor:
IOError: [Errno 9] Bad file descriptor

It is thought that the cause is trying to close the file object (= subprocess.Popen.stdout) that the child process is holding at the timing when the stream is closed (= in this case, when exiting the with statement).