A dozen (or so) ways to start sub-processes in Ruby: Part 2
In the previous article we looked at some basic methods for starting subprocesses in Ruby. One thing all those methods had in common was that they didn’t permit a lot of communication between parent process and child. In this article we’ll examine a few built-in Ruby methods which give us the ability to have a two-way conversation with our subprocesses.
The complete source code for this article can be found at http://gist.github.com/146199.
Method #4: Opening a pipe
As you know, the Kernel#open
method allows you to open files for reading and writing (and, with addition of the open-uri library, HTTP sockets as well). What you may not know is that Kernel.open
can also open processes as if they were files.
puts "4a. Kernel#open with |" cmd = %Q<|#{RUBY} -r#{THIS_FILE} -e 'hello("open(|)", true)'> open(cmd, 'w+') do |subprocess| subprocess.write("hello from parent") subprocess.close_write subprocess.read.split("\n").each do |l| puts "[parent] output: #{l}" end puts end puts "---"
By passing a pipe (“|”) as the first character in the command, we signal to open
that we want to start a process, not open a file. For a command, we’re starting another Ruby process and calling our trusty hello
method (see the first article or the source code for this article for the definition of the hello
method RUBY
and THIS_FILE
constants).
open
yields an IO object which enables us to communicate with the subprocess. Anything written to the object is piped to the process’ STDIN, and the anything the process writes to its STDOUT can be read back as if reading from a file. In the example above we write a line to the child, read some text back from the child, and then end the block.
Note the call to close_write
on line 5. This call is important. Because the OS buffers input and output, it is possible to write to a subprocess, attempt to read back, and wait forever because the data is still sitting in the buffer. In addition, filter-style programs typically wait until they see an EOF on their STDIN to exit. By calling close_write
, we cause the buffer to be flushed and an EOF to be sent. Once the subprocess exits, its output buffer wil be flushed and any read
calls on the parent side will return.
Also note that we pass “w+” as the file open mode. Just as with files, by default the IO object will be opened in read-only mode. If we want to both write to and read from it, we need to specify an appropriate mode.
Here’s the output of the above code:
4a. Kernel#open with | [child] Hello, standard error [parent] output: [child] Hello from open(|) [parent] output: [child] Standard input contains: "hello from parent" ---
Another way to open a command as an IO object is to call IO.popen
:
puts "4b. IO.popen" cmd = %Q<#{RUBY} -r#{THIS_FILE} -e 'hello("popen", true)'> IO.popen(cmd, 'w+') do |subprocess| subprocess.write("hello from parent") subprocess.close_write subprocess.read.split("\n").each do |l| puts "[parent] output: #{l}" end puts end puts "---"
This behaves exactly the same as the Kernel#open
version. Which way you choose to use is a matter of preference. The IO.popen
version arguably makes it a little more obvious what is going on.
Method #5: Forking to a pipe
This is a variation on the previous technique. If Kernel#open
is passed a pipe followed by a dash (“|-“) as its first argument, it starts a forked subprocess. This is like the previous example except that instead of executing a command, it forks the running Ruby process into two processes.
puts "5a. Kernel#open with |-" open("|-", "w+") do |subprocess| if subprocess.nil? # child hello("open(|-)", true) exit else # parent subprocess.write("hello from parent") subprocess.close_write subprocess.read.split("\n").each do |l| puts "[parent] output: #{l}" end puts end end puts "---"
Both processes then execute the given block. In the child process, the argument yielded to the block will be nil
. In the parent, the block argument will be an IO object. As before, the IO object is tied to the forked process’ standard input and standard output streams.
Here’s the output:
5a. Kernel#open with |- [child] Hello, standard error [parent] output: [child] Hello from open(|-) [parent] output: [child] Standard input contains: "hello from parent" ---
Once again, there is an IO.popen
version which does the same thing:
puts "5b. IO.popen with -" IO.popen("-", "w+") do |subprocess| if subprocess.nil? # child hello("popen(-)", true) exit else # parent subprocess.write("hello from parent") subprocess.close_write subprocess.read.split("\n").each do |l| puts "[parent] output: #{l}" end puts end end puts "---"
Applications and Caveats
The techniques we’ve looked at in this article are best suited for “filter” style subprocesses, where we want to feed some input to a process and then use the output it produces. Because of the potential for deadlocks mentioned earlier, they are less suitable for running highly interactive subprocesses which require multiple reads and responses.
open
/popen
also do not give us access to the subprocess’ standard error (STDERR) stream. Any output error generated by the subprocesses will print the same place that the parent process’ STDERR does.
In the upcoming parts of the series we’ll look at some libraries which overcome both of these limitations.
Conclusion
In this article we’ve explored two (or four, depending on how you count it) built-in ways of starting a subprocess and communicating with it as if it were a file. In part 3 we’ll move away from built-ins and on to the facilities provided in Ruby’s Standard Library for starting and controlling subprocesses.
[…] Development, Ruby | Tags: processes, Ruby, shell, subprocesses, terminal, unix | In part 1 and part 2 of this series, we took a look at some of Ruby’s built-in ways to start subprocesses. In this […]
A Dozen (or so) Ways to Start Subprocesses in Ruby: Part 3 « The Devver Blog
April 8, 2010 at 7:55 am
[…] dozen (or so) ways to start sub-processes in Ruby: Part 1 (Part 2, Part […]
The updated Ruby reading list « citizen428.blog()
August 27, 2011 at 9:33 am
[…] Part 2: Opening pipes to processes […]
Generating cows with IO.popen() | Virtuous Code
March 29, 2012 at 6:38 pm