The Devver Blog

A Boulder startup improving the way developers work.

Archive for the ‘Ruby’ Category

A command-line prompt with timeout and countdown

Have you ever started a long operation and walked away from the computer, and come back half an hour later only to find that the process is hung up waiting for some user input? It’s a sub-optimal user experience, and in many cases it can be avoided by having the program choose a default if the user doesn’t respond within a certain amount of time. One example of this UI technique in the wild is powering off your computer – most modern operating systems will pop up a dialogue to confirm or cancel the shutdown, with a countdown until the shutdown proceeds automatically.

This article is about how to achieve the same effect in command-line programs using Ruby.

Let’s start with the end result. We want to be able to call our method like this:

puts ask_with_countdown_to_default("Do you like pie?", 30.0, false)

We pass in a question, a (possibly fractional) number of seconds to wait, and a default value. The method should prompt the user with the given question and a visual countdown. If the user types ‘y’ or ‘n’, it should immediately return true or false, respectively. Otherwise when the countdown expires it should return the default value.

Here’s a high-level implementation:

def ask_with_countdown_to_default(question, seconds, default)
  with_unbuffered_input($stdin) do
    countdown_from(seconds) do |seconds_left|
      write_then_erase_prompt(question, seconds_left) do
        wait_for_input($stdin, seconds_left % 1) do
          case char = $stdin.getc
          when ?y, ?Y then return true
          when ?n, ?N then return false
          else                  # NOOP
          end
        end
      end
    end
  end
  return default
ensure
  $stdout.puts
end                             # ask_with_countdown_to_default

Let’s take it step-by-step.

By default, *NIX terminals operate in “canonical mode”, where they buffer a line of input internally and don’t send it until the user hits RETURN. This is so that the user can do simple edits like backspacing and retyping a typo. This behavior is undesirable for our purposes, however, since we want the prompt to respond as soon as the user types a key. So we need to temporarily alter the terminal configuration.

  with_unbuffered_input($stdin) do

We use the POSIX Termios library, via the ruby-termios gem, to accomplish this feat.

def with_unbuffered_input(input = $stdin)
  old_attributes = Termios.tcgetattr(input)
  new_attributes = old_attributes.dup
  new_attributes.lflag &= ~Termios::ECHO
  new_attributes.lflag &= ~Termios::ICANON
  Termios::tcsetattr(input, Termios::TCSANOW, new_attributes)

  yield
ensure
  Termios::tcsetattr(input, Termios::TCSANOW, old_attributes)
end                             # with_unbuffered_input

POSIX Termios defines a set of library calls for interacting with terminals. In our case, we want to disable some of the terminal’s “local” features – functionality the terminal handles internally before sending input on to the controlling program.

We start by getting a snapshot of the terminal’s current configuration. Then we make a copy for our new configuration. We are interested in two flags: “ECHO” and “ICANON”. The first, ECHO, controls whether the terminal displays characters that the user has types. The second controls canonical mode, which we explained above. After turning both flags off, we set the new configuration and yield. After the block is finished, or if an exception is raised, we ensure that the original terminal configuration is reinstated.

Now we need to arrange for a countdown timer.

    countdown_from(seconds) do |seconds_left|

Here’s the implementation:

def countdown_from(seconds_left)
  start_time   = Time.now
  end_time     = start_time + seconds_left
  begin
    yield(seconds_left)
    seconds_left = end_time - Time.now
  end while seconds_left > 0.0
end                             # countdown_from

First we calculate the wallclock time at which we should stop waiting. Then we begin looping, yielding the number of seconds left, and then when the block returns recalculating the number. We keep this up until the time has expired.

Next up is writing, and re-writing, the prompt.

      write_then_erase_prompt(question, seconds_left) do

This method is implemented as follows:

def write_then_erase_prompt(question, seconds_left)
  prompt_format = "#{question} (y/n) (%2d)"
  prompt = prompt_format % seconds_left.to_i
  prompt_length = prompt.length
  $stdout.write(prompt)
  $stdout.flush

  yield

  $stdout.write("\b" * prompt_length)
  $stdout.flush
end                             # write_then_erase_prompt

We format and print a prompt, flushing the output to insure that it is displayed immediately. The prompt includes a count of the number of seconds remaining until the query times out. In order to make it a nice visually consistent length, we use a fixed-width field for the countdown (“%2d”). Note that we don’t use

puts

to print the prompt – we don’t want it to advance to the next line, because we want to be able to dynamically rewrite the prompt as the countdown proceeds.

After we are done yielding to the block, we erase the prompt in preparation for the next cycle. In order to erase it we create and output string of backspaces (“\b”) the same length as the prompt.

Now we need a way to wait until the user types something, while still periodically updating the prompt.

        wait_for_input($stdin, seconds_left % 1) do

We pass

wait_for_input

an input stream and a (potentially fractional) number of seconds to wait. In this case we only want to wait until the next second-long “tick” so that we can update the countdown. So we pass in the remainder of dividing seconds_left by 1. E.g. if seconds_left was 5.3, we would set a timeout of 0.3 seconds. After 3/10 of a second of waiting for input, the wait would time out, the prompt would be erased and rewritten to show 4 seconds remaining, and then we’d start waiting for input again.

Here’s the implementation of

wait_for_input

:

def wait_for_input(input, timeout)
  # Wait until input is available
  if select([input], [], [], timeout)
    yield
  end
end                             # wait_for_input

We’re using

Kernel#select

to do the waiting. The parameters to

#select

are a set of arrays – one each for input, output, and errors. We only care about input, so we pass the input stream in the first array and leave the others blank. We also pass how long to wait until timing out.

If new input is detected,

select

returns an array of arrays, corresponding to the three arrays we passed in. If it times out while waiting, it returns

nil

. We use the return value to determine whether to execute the given block or note. If there is input waiting we yield to the block; otherwise we just return.

While it takes some getting used to, handling IO timeouts with

select

is safer and more reliable than using the

Timeout

module. And it’s less messy than rescuing

Timeout::Error

every time a read times out.

Finally, we need to read and interpret the character the user types, if any.

          case char = $stdin.getc
          when ?y, ?Y then return true
          when ?n, ?N then return false
          else                  # NOOP
          end

If the user types ‘y’ or ‘n’ (or uppercase versions of the same), we return

true

or

false

, respectively. Otherwise, we simply ignore any characters the user types. Typing characters other than ‘y’ or ‘n’ will cause the loop to be restarted.

Note the use of character literals like

?y

to compare against the integer character code returned by

IO#getc

. We could alternately use

Integer#chr

to convert the character codes into single-character strings, if we wanted.

Wrapping up, we make sure to return the default value should the timeout expire without any user input; and we output a newline to move the cursor past our prompt.

  return default

And there you have it; a yes/no prompt with a timeout and a visual countdown. Static text doesn’t really capture the effect, so rather than include sample output I’ll just suggest that you try the code out for yourself (sorry, Windows users, it’s *NIX-only).

Full source for this article at: http://gist.github.com/148765

Written by avdi

July 16, 2009 at 10:45 pm

A dozen (or so) ways to start sub-processes in Ruby: Part 2

In the previous article we looked at some basic methods for starting subprocesses in Ruby. One thing all those methods had in common was that they didn’t permit a lot of communication between parent process and child. In this article we’ll examine a few built-in Ruby methods which give us the ability to have a two-way conversation with our subprocesses.

The complete source code for this article can be found at http://gist.github.com/146199.

Method #4: Opening a pipe

As you know, the Kernel#open method allows you to open files for reading and writing (and, with addition of the open-uri library, HTTP sockets as well). What you may not know is that Kernel.open can also open processes as if they were files.

  puts "4a. Kernel#open with |"
  cmd = %Q<|#{RUBY} -r#{THIS_FILE} -e 'hello("open(|)", true)'>
  open(cmd, 'w+') do |subprocess|
    subprocess.write("hello from parent")
    subprocess.close_write
    subprocess.read.split("\n").each do |l|
      puts "[parent] output: #{l}"
    end
    puts
  end
  puts "---"

By passing a pipe (“|”) as the first character in the command, we signal to open that we want to start a process, not open a file. For a command, we’re starting another Ruby process and calling our trusty hello method (see the first article or the source code for this article for the definition of the hello method RUBY and THIS_FILE constants).

open yields an IO object which enables us to communicate with the subprocess. Anything written to the object is piped to the process’ STDIN, and the anything the process writes to its STDOUT can be read back as if reading from a file. In the example above we write a line to the child, read some text back from the child, and then end the block.

Note the call to close_write on line 5. This call is important. Because the OS buffers input and output, it is possible to write to a subprocess, attempt to read back, and wait forever because the data is still sitting in the buffer. In addition, filter-style programs typically wait until they see an EOF on their STDIN to exit. By calling close_write, we cause the buffer to be flushed and an EOF to be sent. Once the subprocess exits, its output buffer wil be flushed and any read calls on the parent side will return.

Also note that we pass “w+” as the file open mode. Just as with files, by default the IO object will be opened in read-only mode. If we want to both write to and read from it, we need to specify an appropriate mode.

Here’s the output of the above code:

4a. Kernel#open with |
[child] Hello, standard error
[parent] output: [child] Hello from open(|)
[parent] output: [child] Standard input contains: "hello from parent"

---

Another way to open a command as an IO object is to call IO.popen:

  puts "4b. IO.popen"
  cmd = %Q<#{RUBY} -r#{THIS_FILE} -e 'hello("popen", true)'>
  IO.popen(cmd, 'w+') do |subprocess|
    subprocess.write("hello from parent")
    subprocess.close_write
    subprocess.read.split("\n").each do |l|
      puts "[parent] output: #{l}"
    end
    puts
  end
  puts "---"

This behaves exactly the same as the Kernel#open version. Which way you choose to use is a matter of preference. The IO.popen version arguably makes it a little more obvious what is going on.

Method #5: Forking to a pipe

This is a variation on the previous technique. If Kernel#open is passed a pipe followed by a dash (“|-”) as its first argument, it starts a forked subprocess. This is like the previous example except that instead of executing a command, it forks the running Ruby process into two processes.

  puts "5a. Kernel#open with |-"
  open("|-", "w+") do |subprocess|
    if subprocess.nil?             # child
      hello("open(|-)", true)
      exit
    else                        # parent
      subprocess.write("hello from parent")
      subprocess.close_write
      subprocess.read.split("\n").each do |l|
        puts "[parent] output: #{l}"
      end
      puts
    end
  end
  puts "---"

Both processes then execute the given block. In the child process, the argument yielded to the block will be nil. In the parent, the block argument will be an IO object. As before, the IO object is tied to the forked process’ standard input and standard output streams.

Here’s the output:

5a. Kernel#open with |-
[child] Hello, standard error
[parent] output: [child] Hello from open(|-)
[parent] output: [child] Standard input contains: "hello from parent"

---

Once again, there is an IO.popen version which does the same thing:

  puts "5b. IO.popen with -"
  IO.popen("-", "w+") do |subprocess|
    if subprocess.nil?             # child
      hello("popen(-)", true)
      exit
    else                        # parent
      subprocess.write("hello from parent")
      subprocess.close_write
      subprocess.read.split("\n").each do |l|
        puts "[parent] output: #{l}"
      end
      puts
    end
  end
  puts "---"

Applications and Caveats

The techniques we’ve looked at in this article are best suited for “filter” style subprocesses, where we want to feed some input to a process and then use the output it produces. Because of the potential for deadlocks mentioned earlier, they are less suitable for running highly interactive subprocesses which require multiple reads and responses.

open/popen also do not give us access to the subprocess’ standard error (STDERR) stream. Any output error generated by the subprocesses will print the same place that the parent process’ STDERR does.

In the upcoming parts of the series we’ll look at some libraries which overcome both of these limitations.

Conclusion

In this article we’ve explored two (or four, depending on how you count it) built-in ways of starting a subprocess and communicating with it as if it were a file. In part 3 we’ll move away from built-ins and on to the facilities provided in Ruby’s Standard Library for starting and controlling subprocesses.

Written by avdi

July 13, 2009 at 4:29 pm

Posted in Ruby, Tips & Tricks

Tagged with ,

Devver adds Postgres and SQLite database support

We are working hard to quickly expand our compatibility on Ruby projects. With that goal driving us, we are happy to announce support for Postgres and SQLite databases. With the addition of these database options, along with our existing support for MySQL, Devver now supports all of the most popular databases commonly used with Ruby. These three databases are the default databases tested against ActiveRecord and we expect will cover the majority of the Ruby community.

To begin working with Postgres or SQLite on Devver all you need to do is have a database.yml with the test environment set to the adapter of your choice. If we don’t support your favorite database, you can still request a beta invite and let us know which database you want us to support. If we just added support for your database, perhaps we can speed up your project on Devver, so request a beta invite.

Written by DanM

July 6, 2009 at 12:24 pm

Posted in Development, Devver, Ruby, Testing

Tagged with , ,

A dozen (or so) ways to start sub-processes in Ruby: Part 1

Introduction

It is often useful in Ruby to start a sub-process to run a particular chunk of Ruby code. Perhaps you are trying to run two processes in parallel, and Ruby’s green threading doesn’t provide sufficient concurrency. Perhaps you are automating a set of scripts. Or perhaps you are trying to isolate some untrusted code while still getting information back from it.

Whatever the reason, Ruby provides a wealth of facilities for interacting with sub-processes, some better known than others. In this series of articles I will be focusing on running Ruby as a sub-process of Ruby, although many of the techniques I’ll be demonstrating are applicable to running any type of program in a sub-process. I’ll also be keeping the focus on UNIX-style platforms, such as Linux and Mac OS X. Sub-process handling on Windows differs significantly, and we’ll leave that for another series.

In the first and second articles, I’ll demonstrate some of the facilities for starting sub-processes that Ruby possesses out-of-the-box, no requires needed. In the third article we’ll look at some tools provided in Ruby’s Standard Library which build on the methods introduced in part one. And in the fourth instalment I’ll briefly survey a few of the many Rubygems which simplify sub-process interactions.

Getting Started

To begin, let’s define a few helper methods and constants which we’ll refer back to throughout the series. First, let’s define a simple method which will serve as our “slave” code – the code we want to execute in a sub-process. Here it is:

def hello(source, expect_input)
  puts "[child] Hello from #{source}"
  if expect_input
    puts "[child] Standard input contains: \"#{$stdin.readline.chomp}\""
  else
    puts "[child] No stdin, or stdin is same as parent's"
  end
  $stderr.puts "[child] Hello, standard error"
end

(Note: The full source code for this article can be found at http://gist.github.com/137705)

This method prints a message to the standard output stream, a message to the standard error stream, and optionally reads and prints a message from the standard input stream. One of the things we’ll be exploring in this series is the differing ways in which the various sub-process-starting methods handle standard I/O streams.

Next, let’s define a couple of helpful constants.

require 'rbconfig'
THIS_FILE = File.expand_path(__FILE__)

RUBY = File.join(Config::CONFIG['bindir'], Config::CONFIG['ruby_install_name'])

The first, THIS_FILE, is simply the fully-qualified name of the file containing our demo source code. RUBY, the second constant, is set to the fully-qualified path of the running Ruby executable. These constants will come in handy with sub-process methods which require an explicit shell command to be run.

In order to make the order of events clearer, we’ll force the standard output stream into synchronised mode. This will cause it to flush its buffer after every write.

$stdout.sync = true

Finally, we’ll be surrounding all of the code which follows in the following protective IF-statement:

if $PROGRAM_NAME == __FILE__
# ...
end

This will ensure that the demo code won’t be re-executed when we require the source file within sub-processes.

Method #1: The Backtick Operator

The simplest way to execute a sub-process in Ruby is with the backtick (]`). This method, which harks back to Bourne Shell scripting and Perl, is concise and often gives us exactly as much interaction as we need with a sub-process. The backtick, while it may look like a part of Ruby’s core syntax, is technically an operator defined by Kernel. Like most Ruby operators it can be redefined in your own code, although that’s beyond the scope of this article. Kernel defines the backtick operator as a method which executes its argument in a subshell.

puts "1. Backtick operator"
output = `#{RUBY} -r#{THIS_FILE} -e'hello("backticks", false)'`
output.split("\n").each do |line|
  puts "[parent] output: #{line}"
end
puts

Here, we use backticks to execute a child Ruby process which loads our demo source code and executes the hello method. This yields:

1. Backtick operator
[child] Hello, standard error
[parent] output: [child] Hello from backticks
[parent] output: [child] No stdin, or stdin is same as parent's

The backtick operator doesn’t return until the command has finished. The sub-process inherits its standard input and standard error streams from the parent process. The process’ ending status is made available as a Process::Status object in the $? global (aka $CHILD_STATUS if the English library is loaded).

We can use the %x operator as an alternate syntax for backticks, which enables us to select arbitrary delimiters for the command string. E.g. %x{echo `which cowsay`}.

Method #2: Kernel#system

Kernel#system is similar to the backtick operator in operation, with one important difference. Where the backtick operator returns the STDOUT of the finished command, system returns a Boolean value indicating the success or failure of the command. If the command exits with a zero status (indicating success), system will return true. Otherwise it returns false.

puts "2. Kernel#system"
success = system(RUBY, "-r", THIS_FILE, "-e", 'hello("system()", false)')
puts "[parent] success: #{success}"
puts

This results in:

2. Kernel#system
[child] Hello from system()
[child] No stdin, or stdin is same as parent's
[child] Hello, standard error
[parent] success: true

Just like the backtick operator, system doesn’t return until its process has exited, and leaves the process exit status in $?. The sub-process inherits the parent process’ standard input, output, and error streams.

As we can see in the example above, when system() is given multiple arguments they are assembled into a single command for execution. This feature can make system() a little more convenient than backticks for executing complex commands. For this reason and because it’s more visually apparent in the code, I prefer to use Kernel#system over backticks unless I need to capture the command’s output. Note that there are some other ways system() can be called; see the Kernel#exec documentation for the details.

Method #3: Kernel#fork (aka Process.fork)

Ruby provides access to the *NIX fork() system call via Kernel#fork. On UNIX-like OSes, fork splits the currently executing Ruby process in two. Both processes run concurrently and independently from that point on. Unlike the methods we’ve examined so far, fork enables us to execute in-line Ruby code in a sub-process, rather than explicitly starting a new Ruby interpreter and telling it to load our code.

Traditionally we would need to put in some conditional code to examine the return value of fork and determine whether the code was executing in the parent or child process. Ruby makes it easy to specify what code should be run in the child by allowing us to pass a block to fork. The contents of the block will be run in the child process, after which it will exit. The parent will continue running at the point where the block ends.

puts "3. Kernel#fork"
pid = fork do
hello("fork()", false)
end
Process.wait(pid)
puts "[parent] pid: #{pid}"
puts

This produces the following output:

3. Kernel#fork
[child] Hello from fork()
[child] No stdin, or stdin is same as parent's
[child] Hello, standard error
[parent] pid: 19935

Note the call to Process.wait. Since the process spawned by fork runs concurrently with the parent process, we need to explicitly wait for the child process to finish if we want to synchronize with it. We use the child process ID, returned by fork, as the argument to Process.wait.

The sub-process inherits its standard error and output streams from the parent. Since fork is a *NIX-only syscall, it will only reliably work on UNIX-style systems.

Conclusion

In this first installment in the Ruby Sub-processes series we’ve looked at three of the simplest ways to start another Ruby process from inside a Ruby program. Stay tuned for part 2, in which we’ll delve into some methods for doing more complex communication with spawned sub-processes.

Written by avdi

June 30, 2009 at 8:57 am

Posted in Ruby, Tips & Tricks

Tagged with ,

SimpleDB DataMapper Adapter: Progress Report

From the beginning of Devver, we decided we wanted to work with some new technologies and we wanted to be able to scale easily. After looking at options AWS seemed to have many technologies that could help us build and scale a system like Devver. One of these technologies was SimpleDB. One of the other new things we decided to try was DataMapper (DM) rather than the more familiar ActiveRecord. This eventually let me to work on my own SimpleDB DataMapper adapter.

Searching for ways to work with SDB using Ruby, we found a SimpleDB DM adapter by Jeremy Boles. It worked well initially but as our needs grew (and to make it compatible with the current version of DM) it became necessary to add and update the features of the adapter. These changes lived hidden in our project’s code for awhile, for no other reason than we were too lazy to really commit it all back on GitHub. Recently though there has been a renewed interest about working with on SimpleDB with Ruby. I started pushing the code updates on GitHub, then I got a couple requests and suggestions here and there to improve the adapter. One of these suggestions cam from Ara Howard, who is doing impressive work of his own on Ruby and AWS, specifically SimpleDB. His suggestion on moving from the aws_sdb gem to right_aws, which along with other changes improved performance significantly (1.6x on write, up to 36x on reading large queries over the default limit of 100 objects). Besides performance improvements, we have recently added limit and sorting support to the adapter.

#new right_aws branch using AWS select
$ ruby scripts/simple_benchmark.rb
      user     system      total        real
creating 200 users
 1.020000   0.240000   1.260000 ( 35.715608)
Finding all users age 25 (all of them), 100 Times
 59.280000   8.640000  67.920000 ( 99.727380)

#old aws_sdb using query with attributes
$ ruby scripts/simple_benchmark.rb
      user     system      total        real
creating 200 users
  1.290000   0.530000   1.820000 ( 52.916103)
Finding all users age 25 (all of them), 100 Times
  356.640000  53.090000 409.730000 (3574.260988)

view this gist

As I added features, testing the adapter also became slow, (over a minute a run) because the functional tests actually connect to and use SimpleDB. Since Devver is all about speeding up Ruby tests, I decided to get the tests running on Devver. It was actually very easy and sped up the test suite from 1 minute and 8 seconds down to 28 seconds. You can check out how much Devver speeds up the results yourself.

We are currently using the SimpleDB adapter to power our Devver.net website as well as the Devver backend service. It has been working well for us, but we know that it doesn’t cover everyone’s needs. Next time you are creating a simple project, give SimpleDB a look, we would love feedback about the DM adapter, and it would be great to get some other people contributing to the project. If anyone does fork my SDB Adapter Github repo, feel free to send me pull requests. Also, let me know if you want to try using Devver as you hack on the adapter, it can really speed up testing, and I would be happy to give out a free account.

Lastly, at a recent Boulder Ruby users group meet up, the group did a code review for the adapter. It went well and I should finish cleaning up the code and get the improvements suggested by the group committed to GitHub soon.

Update: The refactorings suggested at the code review are now live on GitHub.

Written by DanM

June 22, 2009 at 11:27 am

Spellcheck your files with Aspell and Rake

We recently redid our website. The new site included a new design and much more content explaining what we do. We wanted a quick way to check over everything and make sure we didn’t miss any spelling errors or typos. First I started looking for a web service that could scan the site for spelling errors. I found spellr.us, which is nice but would only catch errors once they were live. It also can’t scan all of the pages which require being logged in.

I was pairing with Avdi who thought we should just run Aspell, which worked out great. We were originally trying to just create a simple Emacs macro to go through all our HTML files and check them but in the end created simple Rake tasks, which makes it really easy to integrate spellcheck into CI. After Avdi figured out the commands we needed to use on each file to get the information we needed from Aspell, it was easy to just wrap the command using Rake’s FileList. To keep everyone on the same setup, we created a local dictionary of words to ignore or accept and keep that checked into source control as well.

The final solution grabs all the files you want to spell check, then runs them through Aspell with HTML filtering. We have two tasks: one that runs in interactive mode the the user can fix mistakes and one mode for CI that just fails if it finds any errors.

def run_spellcheck(file,interactive=false)
  if interactive
    cmd = "aspell -p ./config/devver_dictionary -H check #{file}"
    puts cmd
    system(cmd)
    [true,""]
  else
    cmd = "aspell -p ./config/devver_dictionary -H list  'spellcheck:interactive'

namespace :spellcheck do
  files = FileList['app/views/**/*.html.erb']

  desc "Spellcheck interactive"
  task :interactive do
    files.each do |file|
      run_spellcheck(file,true)
    end
    puts "spelling check complete"
  end

  desc "Spellcheck for ci"
  task :ci do
    files.each do |file|
      success, results = run_spellcheck(file)
      unless success
        puts results
        exit 1
      end
    end
    puts "no spelling errors"
    exit 0
  end
end

view this gist

Written by DanM

May 26, 2009 at 8:33 am

Single-file Sinatra apps with specs baked-in

It’s so easy to create little single-file apps in Sinatra that it almost seems a shame to start a second file just for tests.  The other day Dan and I decided to see if we could create a Sinatra app with everything – including the tests – baked right in.  Here’s what we came up with.

#!/usr/bin/env ruby
require 'rubygems'
gem 'rack', '=0.9.1'
gem 'thin', '=1.0.0'
require 'sinatra'

get '/' do
  content_type 'text/plain'
  "Hello, world"
end

# Run me with 'spec' executable to run my specs!
if $0 =~ /spec$/
  set :environment, :test
  set :run,         false       # Don't autostart server

  require 'spec/interop/test'
  require 'sinatra/test'

  describe "Example App" do
    include Sinatra::Test

    it "should serve a greeting" do
      get '/'
      response.should be_ok
      response.body.should == "Hello, world"
    end

    it "should serve content as text/plain" do
      get '/'
      response.headers['Content-Type'].should == 'text/plain'
    end

  end
end

view this gist
The code switches modes on the name of the executable used to run the file. If we run it with the

spec

command, we get a test run:

$ spec -fs sinatra-tests-baked-in.rb

Example App
- should serve a greeting
- should serve content as text/plain

Finished in 0.007221 seconds

2 examples, 0 failures

Otherwise, if we call it as a Ruby program, it runs the Sinatra server as we would expect:

$ ruby sinatra-tests-baked-in.rb
== Sinatra/0.9.1.1 has taken the stage on 4567 for development with backup from Thin
>> Thin web server (v1.0.0 codename That's What She Said)
>> Maximum connections set to 1024
>> Listening on 0.0.0.0:4567, CTRL+C to stop

And there you have it: a true single-file application, specs and all.

Written by avdi

May 13, 2009 at 9:00 am

Posted in Development, Hacking, Ruby

Tagged with , ,

Managing Amazon EC2 with your iPhone

I wanted a quick way when out and about to easily manage our AWS EC2 instances while out and about. It hasn’t happened often, but occasionally I am away from the computer and I need to reboot the instances. Perhaps I remember our developer cluster isn’t being used and want to shut it down to save some money.

I didn’t find anything simple and free with a quick Google search, so in a about an hour I wrote a nice little Sinatra app that will let me view our instances, shutdown, or reboot any specific instance or all of them. The tiny framework actually turned out to be even more useful as I now have options that let us tail error logs, reboot Apache, reboot mongrel clusters, or execute any common system administration task.

I won’t be going into detail on how to build a iPhone webapp using Sinatra and iUI, because Ben already created an excellent post detailing all of those steps. In fact I used his old project as the template when I created this project. I can’t begin to explain how amazingly simple it is to build an iPhone webapp using Sinatra, so if you have been thinking of a quick project I highly recommend it.

Here are some screen shots showing the final app. (screenshot courtesy of iPhoney):

ec2 manager home view

ec2 manager home view.

ec2 manager describe view

ec2 manager describe instances view.

ec2 manager instance view.

ec2 manager instance view.

This app uses the Amazon EC2 API Tools to do all the heavy lifting. So this app assumes that you already have the tools installed and working on the machine you want this app to run on. This normally involves installing the tools and setting up some environment variables like EC2_HOME, so make sure you can run ec2-describe-instances from the machine. After that you should just have to change EC2_HOME in the Sinatra app to match the path where you installed the EC2 tools.

Let me know if you have any issues, it is quick and dirty, but I have already found it useful.

To run the app:
cmd> ruby -rubygems ./ec2_manager.rb

require 'sinatra'

EC2_HOME = '~/.ec2'

use Rack::Auth::Basic do |username, password|
  [username, password] == ['some_user', 'some_pass']
end

get "/" do
  @links = %w{describe_ec2s restart_all_ec2s shutdown_all_ec2s}.map { |cmd|
    cmd_link(cmd)
  }.join
  erb :index
end

get "/describe_ec2s" do
  results = `cd #{EC2_HOME}; ec2-describe-instances`
  instances = results.scan(/INSTANCE\ti-\w*/).each{|i| i.sub!("INSTANCE\t",'')}
  @links = instances.map { |i|
    instance_link(i)
  }.join
  erb :index
end

get "/restart_all_ec2s" do
  @results = `cd #{EC2_HOME}; ec2-describe-instances`
  instances = @results.scan(/INSTANCE\ti-\w*/).each{|i| i.sub!("INSTANCE\t",'')}
  cmd="cd #{EC2_HOME}; ec2-reboot-instances #{instances.join(' ')}"
  @results = `cmd`
  erb :index
end

get "/shutdown_all_ec2s" do
  @results = `cd #{EC2_HOME}; ec2-describe-instances`
  instances = @results.scan(/INSTANCE\ti-\w*/).each{|i| i.sub!("INSTANCE\t",'')}
  cmd="cd #{EC2_HOME}; ec2-terminate-instances #{instances.join(' ')}"
  @results = `cmd`
  erb :index
end

get "/instance/:id" do
  id = params[:id] if params[:id]
  verify_id(id)
  @results = `cd #{EC2_HOME}; ec2-describe-instances #{id}`
  @links = "<li><a href='/shutdown/#{id}' target='_self'>shutdown #{id}</a></li>"
  @links += " <li><a href='/reboot/#{id}' target='_self'>reboot #{id}</a></li>"
  erb :index
end

get "/reboot/:id" do
  id = params[:id] if params[:id]
  verify_id(id)
  @results = `cd #{EC2_HOME}; ec2-reboot-instances #{id}`
  erb :index
end

get "/shutdown/:id" do
  id = params[:id] if params[:id]
  verify_id(id)
  @results = `cd #{EC2_HOME}; ec2-terminate-instances #{id}`
  erb :index
end

helpers do

  def cmd_link(cmd)
    "<li><a href='#{cmd}' target='_self'>#{cmd}</a></li>"
  end

  def instance_link(instance)
    "<li><a href='/instance/#{instance}' target='_self'>#{instance}</a></li>"
  end

  def verify_id(id)
    raise Sinatra::ServerError, 'bad-id, What you doin?' unless id.match(/i-\w*/)
  end

end

use_in_file_templates!

__END__

@@ index



@import "/stylesheets/iui.css";




<div class="toolbar">
<h1 id="pageTitle"></h1>
</div>


<ul id="home">
<li><a href='/' target='_self'>home</a></li>


</ul>





<li><strong>results</strong></li>

<ul id="home">
<li><a href='/' target='_self'>home</a></li>

&lt;%= @results.gsub(&quot;\n&quot;,&quot;<br />") %&gt;
</ul>




view this gist

Written by DanM

March 5, 2009 at 10:03 am

Using Ruby to Send Update Emails to Our Mentors

At Devver.net, we send out weekly email updates to an awesome set of mentors. We do this for a number of reasons. First and foremost, we get valuable feedback and advice from our mentors on a variety of issues. But it’s also an easy and effective way to keep us on track and even maximize our chances of success. As Paul Graham says in How Not To Die (he was talking directly to YC teams, but you’ll get the idea):

“For us the main indication of impending doom is when we don’t hear from you. When we haven’t heard from, or about, a startup for a couple months, that’s a bad sign.

Maybe if you can arrange that we keep hearing from you, you won’t die.

That may not be so naive as it sounds. … [The] mere constraint of staying in regular contact with us will push you to make things happen, because otherwise you’ll be embarrassed to tell us that you haven’t done anything new since the last time we talked.”

Foodzie started emailing their mentors early in the summer. We actually borrowed (stole) their email format and best practices.

One thing we’ve tried to not do is send out a completely generic email to all our mentors. Depending on the content and the interaction we’ve had with a specific mentor, we’ll adjust his email accordingly. We begin each email with their name and send it directly to them (in other words, we don’t put a huge list of addresses in the To, CC, or BCC fields). We do this because we can tailor it and it helps elicit individual responses from each mentor (it’s easier to ignore a question if it’s sent to a group).

But, of course, sometimes the emails to a few mentors can be identical. In this case, my not-so-well-kept secret is that I just use a simple Ruby script to send out a duplicate email that appears to be hand-crafted (or at least copied and pasted).

I’ve been told that Outlook can perform this functionality easily, but I don’t know of any way to do this within Gmail. If there is, let me know so I can feel a little silly (in any case, the Ruby code was fun to write).

To run this code, you’ll need to install the highline gem. You’ll also need to add your Gmail account, recipients, subject message, etc. Finally, you’ll want to put your message inside a separate file within project directory. That way, you can easily modify, spellcheck, and format to your heart’s content before sending.

You can get the entire gmailr source code (all two files!) at Github. Please use this script for good, not evil – no one likes a spammer. Enjoy!

Written by Ben

January 20, 2009 at 3:46 pm

Ruby people on Twitter

The Ruby community is always quickly moving, changing, and adopting new things. It is good to keep your ear to the ground so you can learn and adopt things that the community is finding really useful. There are a number of ways to do this, like watching the most popular Ruby projects on GitHub, most active projects on RubyForge, Ruby Reddit, or listening to the Rails podcast. The way I have found most effective is following a good collection of the Ruby community on Twitter, many of the most active Ruby community members and companies are on Twitter. It is where I have first heard of many things going on in Ruby like the recent Merb/Rails merge.

You can find a great list of 50+ (now 100+) Rubyists to follow on Twitter from RubyLearning. I thought we might as well give out a list of some of the Ruby people Devver.net is following on twitter.

technoweenie
jamis / Jamis Buck
obie / Obie Fernandez
chadfowler / Chad Fowler
engineyard / Engine Yard
d2h / DHH
rjs / Ryan Singer
jasonfried / Jason Fried
37signals
foodzie
fiveruns
_why / why the lucky stiff
gilesgoatboy / Giles Bowkett
dlsspy / Dustin Sallings
julien51 / julien
rbates / Ryan Bates
defunkt / Chris Wanstrath
chrismatthieu / Chris Matthieu
littleidea / Andrew Clay Shafer
headius / Charles Nutter
bascule / Tony Arcieri
atmos / Corey Donohoe
ubermajestix / Tyler Montgomery
raganwald / Reg Braithwaite
chriseppstein


of course I have to give a special shout to ourselves:

danmayer / Dan Mayer
bbrinck / Ben Brinckerhoff
devver.net

If we should be following you also send us an email at contact@devver.net, and we can hook up on Twitter as well.

Written by DanM

January 8, 2009 at 2:08 pm

Posted in Misc, Ruby, Tips & Tricks

Follow

Get every new post delivered to your Inbox.