[Ruby] Different rules for buffering and flushing depending on whether standard output is a console or another command

3 minute read

Environment

  • ruby 2.7.1p83 (2020-03-31 revision a0c7c23c9c) [x86_64-linux]
  • Ubuntu 20.04 LTS (Focal Fossa) [Windows Subsystem for Linux 2; WSL2]

Event

When using standard output with Ruby, it seems that the rules for buffering flush differ depending on whether the standard output is to the console or another command.

  • Buffer if standard output is connected to another command
  • Flush line by line if standard output is connected to the console.

Let’s verify this easily. Ruby code that outputs true if the standard output is connected to the console and false if it is connected to the command, and then outputs the numbers 0 to 4 to the standard output every 1 second. is.

main.rb


puts "STDOUT.isatty = #{STDOUT.isatty}"
5.times do |x|
  sleep 1
  puts x
end

The following cmd.rb and console.rb execute the standard output of ruby main.rb by connecting it to a pipe or console, and add a timestamp to the output result.

cmd.rb


require'open3'

# Read line by line from read-only IO, add timestamp and output.
puts'*** If the standard output of main.rb is connected to another command ***'
Open3.popen3('ruby main.rb') do |stdin, stdout, stderr, wait_thr|
  stdout.each do |line|
    timestamp = Time.now.strftime('%F %T')
    puts "#{timestamp} #{line}"
  end
end

console.rb


require'pty'

# Read line by line from read-only IO, add timestamp and output.
puts'*** If the standard output of main.rb is connected to the console ***'
PTY.spawn('ruby main.rb') do |read, write, pid|
  read.each do |line|
    timestamp = Time.now.strftime('%F %T')
    puts "#{timestamp} #{line}"
  end
end

The execution results of cmd.rb and console.rb are as follows.

$ ruby cmd.rb 2>/dev/null
*** When the standard output of main.rb is connected to another command ***
2020-07-05 16:45:09 STDOUT.isatty = false
2020-07-05 16:45:09 0
2020-07-05 16:45:09 1
2020-07-05 16:45:09 2
2020-07-05 16:45:09 3
2020-07-05 16:45:09 4
$ ruby console.rb 2>/dev/null
*** When the standard output of main.rb is connected to the console ***
2020-07-05 16:45:11 STDOUT.isatty = true
2020-07-05 16:45:12 0
2020-07-05 16:45:13 1
2020-07-05 16:45:14 2
2020-07-05 16:45:15 3
2020-07-05 16:45:16 4

If you look at the timestamp, you can see that the standard output is piped to main.rb, but if it’s connected to the console, it’s flushing line by line.

Do not want to buffer even if standard output leads to a command

I think there are various ways to do it, but is it the one that comes to mind for now?

  • Set standard output to sync mode by setting STDOUT.sync = true
  • Call STDOUT.flush at any time.

Let’s experiment with the former. Modify the above main.rb as follows.

main.rb


$stdout.sync = true
puts "STDOUT.isatty = #{STDOUT.isatty}"
5.times do |x|
  sleep 1
  puts x
end

If you execute cmd.rb which connects the standard output of main.rb to another command, you can see that it is not buffering unlike the previous one.

$ ruby cmd.rb 2>/dev/null
*** When the standard output of main.rb is connected to another command ***
2020-07-05 16:51:33 STDOUT.isatty = false
2020-07-05 16:51:34 0
2020-07-05 16:51:35 1
2020-07-05 16:51:36 2
2020-07-05 16:51:37 3
2020-07-05 16:51:38 4

Don’t want to flush, even if standard output goes to the console

There are various ways to do this, but if it is within the range you can think of, “Redirect $stdout to StringIO and output the content accumulated in StringIO to STDOUT at any timing” There is one. In this case, main.rb should look like this:

main.rb


require'stringio'

StringIO.open do |io|
  # Redirect standard output to StringIO
  $stdout = io

  # Here is the same as the original main.rb
  puts "STDOUT.isatty = #{STDOUT.isatty}"
  5.times do |x|
    sleep 1
    puts x
  end

  # The contents stored in StringIO are discharged to STDOUT
  $stdout = STDOUT
  io.rewind
  puts io.read
end

If you execute console.rb and check the behavior when main.rb is connected to the console, you can see that it is buffering as expected.

$ ruby console.rb 2>/dev/null
*** When the standard output of main.rb is connected to the console ***
2020-07-05 16:56:38 STDOUT.isatty = true
2020-07-05 16:56:38 0
2020-07-05 16:56:38 1
2020-07-05 16:56:38 2
2020-07-05 16:56:38 3
2020-07-05 16:56:38 4

Tags:

Updated: