How to pass the execution result of a shell command in a list in Python (non-blocking version)

In How to pass the execution result of shell command in Python as a list, there was such a source. This is a good idea, but it's a shame to wait for the shell command to finish.

res_cmd_no_lfeed.py

#!/usr/bin/python

import subprocess

def res_cmd_lfeed(cmd):
  return subprocess.Popen(cmd, stdout=subprocess.PIPE, shell=True).stdout.readlines()

def res_cmd_no_lfeed(cmd):
  return [str(x).rstrip("\n") for x in res_cmd_lfeed(cmd)]

def main():
  cmd = ("ls -l")
  print(res_cmd_no_lfeed(cmd))

if __name__ == '__main__':
  main()

So use yield

res_cmd_no_lfeed.py

#!/usr/bin/python

import subprocess

def res_cmd_lfeed(cmd):  #The return value is not a list but each line
  for line in subprocess.Popen(cmd, stdout=subprocess.PIPE,shell=True).stdout:
      yield line

def res_cmd_no_lfeed(cmd):
  return [str(x).rstrip("\n") for x in res_cmd_lfeed(cmd)]

def main():
  cmd = ("ls -l")
  print(res_cmd_no_lfeed(cmd))

if __name__ == '__main__':
  main()

res_cmd_lfeed does not block until the end of the command, but returns a yield when it receives a line in the loop, that is, it stops the loop and returns the result at that time. If you are told "Next," it will resume and return the next result ... and so on.

In this example, res_cmd_no_lfeed waits until all the Arrays are completed, so it doesn't make sense. If it's a web application, you can steadily return a response to the browser.

What are you happy about

• Even if the shell command produces infinite output, you can receive the result halfway.
• Even if it takes a long time to execute a shell command, you can receive the result halfway early.
• Working memory is minimal because the target to work at one time is small.
• Even if it fails in the second half, it is irreversible because it returns the result halfway. unhappy

forevertime.bat

@echo off
:TOP
echo %time%
goto top

You can see it by doing something like this. Since this batch file keeps outputting indefinitely, the former source keeps waiting with readlines (), but the latter captures and processes the result more and more. Like "Tear off and throw".

Caution. The pipe has a buffer

• If you do not receive the result of the shell command, the output of the shell command may get stuck and stop. When I tried it on Windows 10, it seems that there is a buffer of about 12 kb. If you take it more leisurely, you can observe it as a phenomenon that the time stamp flies by 10 seconds.

forevertime.bat(Mass output version)

@echo off
:TOP
set /P X=0123456789012345678901234567890123456789012345678901234567890123<NUL
set /P X=0123456789012345678901234567890123456789012345678901234567890123<NUL
set /P X=0123456789012345678901234567890123456789012345678901234567890123<NUL
set /P X=0123456789012345678901234567890123456789012345678901234567890123<NUL
set /P X=0123456789012345678901234567890123456789012345678901234567890123<NUL
set /P X=0123456789012345678901234567890123456789012345678901234567890123<NUL
set /P X=0123456789012345678901234567890123456789012345678901234567890123<NUL
set /P X=0123456789012345678901234567890123456789012345678901234567890123<NUL
echo %time%
goto top

I think I can tune more and more, but I wonder if I should leave it here.