The Devver Blog

A Boulder startup improving the way developers work.

Ruby Tools Roundup

Update: Devver now offers a hosted metrics service for Ruby developers which can give you useful feedback about your code. Check out Caliper, to get started with metrics for your project.

I collected all of the Ruby tools posts I made this week into a single roundup. You can quickly jump to any tool that interests you or read my reviews start to finish. If you just want to read a individual section here are the previous posts Ruby Code Quality Tools, Ruby Test Quality Tools, and Ruby Performance Tools.

There have been a bunch of interesting tools released for Ruby lately. I decided to write about a few of my favorite Ruby tools and give some of the new tools a shot as well. Simply put, better tools can help you be a better developer. I am ignoring the entire topic of IDEs as tools, as I have written about Ruby IDEs before, and it is basically a religious war. If you use any Ruby tools I don’t mention be sure to let me know as I am always interested in trying something new out.

Tool Name Description
Code Quality Tools
Roodi Roodi gives developers information about common mistakes in their Ruby code. It makes it easy to clean up your code before things start to get ugly.
Dust Dust is a new tool that will analyze your code, detect unsafe blocks and unused code. Dust is being created by the same mind behind Heckle
Flog Flog essentially scores an ABC metric, giving you a good understanding of the overall code complexity of any give file or method.
Saikuro When given Ruby source code, Saikuro will generate a report listing the cyclomatic complexity of each method found.
Test Quality Tools
Heckle Heckle helps test your Ruby tests (how cool is that?). Heckle is a mutation tester. It alters/breaks code and verifies that tests fail.
rcov rcov is the easiest way to get information about your current code coverage.
Ruby/Rails Performance Tools
ruby-prof ruby-prof is a fast and easy-to-use Ruby profiler. The first of four tools that can help you solve performance issues.
New Relic New Relic is one of the three Rails plugin performance debugging and monitoring tools recently released.
TuneUp TuneUp a Rails performance tool from FiveRuns. This tool has an interesting community built around it as well.
RubyRun Ruby Run is a Rails performance tool similar to New Relic and TuneUp

Lets get into it…

Roodi


Roodi gives you a bunch of interesting warnings about your Ruby code. We are about to release some code, so I took the opportunity to fix up anything Roodi complained about. It helped identify refactoring opportunities, both with long methods, and overly complex methods. The code and tests became cleaner and more granular after breaking some of the methods down. I even found and fixed one silly performance issue that was easy to see after refactoring, which improved the speed of our code. Spending some time with Roodi looks like it could easily improve the quality and readability of most Ruby projects with very little effort. I didn’t solve every problem because in one case I just didn’t think the method could be simplified anymore, but the majority of the suggestions were right on. Below is an example session with Roodi

dmayer$ sudo gem install roodi
dmayer$ roodi lib/client/syncer.rb
lib/client/syncer.rb:136 - Block cyclomatic complexity is 5.  It should be 4 or less.
lib/client/syncer.rb:61 - Method name "excluded" has a cyclomatic complexity is 10.  It should be 8 or less.
lib/client/syncer.rb:101 - Method name "should_be_excluded?" has a cyclomatic complexity is 9.  It should be 8 or less.
lib/client/syncer.rb:132 - Method name "find_changed_files" has a cyclomatic complexity is 10.  It should be 8 or less.
lib/client/syncer.rb:68 - Rescue block should not be empty.
lib/client/syncer.rb:61 - Method name "excluded" has 25 lines.  It should have 20 or less.
lib/client/syncer.rb:132 - Method name "find_changed_files" has 27 lines.  It should have 20 or less.
Found 7 errors.

After Refactoring:

~/projects/gridtest/trunk dmayer$ roodi lib/client/syncer.rb
lib/client/syncer.rb:148 - Block cyclomatic complexity is 5.  It should be 4 or less.
lib/client/syncer.rb:82 - Rescue block should not be empty.
Found 2 errors.

I did have one problem with Roodi – the errors about rescue blocks just seemed to be incorrect. For code like the little example below it kept throwing the error even though I obviously am doing some work in the rescue code.

Roodi output: lib/client/syncer.rb:68 - Rescue block should not be empty.
begin
  socket = TCPSocket.new(server_ip,server_port)
  socket.close
  return true
rescue Errno::ECONNREFUSED
  return false
end

Dust


Dust detects unused code like unused variables,branches, and blocks. I look forward to see how the project progresses. Right now there doesn’t seem to be much out there on the web, and the README is pretty bare bones. Once you can pass it some files to scan, I think this will be something really useful. For now I didn’t think there wasn’t much I could actually do besides check it out. Kevin, who also helped create the very cool Heckle, does claim that code scanning is coming soon, so I look forward to doing a more detailed write up eventually.

Flog


Flog gives feedback about the quality of your code by scoring code using the ABC metric. Using Flog to help guide refactoring, code cleanup, and testing efforts can be highly effective. It is a little easier to understand the reports after reading how Flog scores your code, and what is a good Flog score. Once you get used to working with Flog you will likely want to run it often against your whole project after making any significant changes. There are two easy ways to do this a handy Flog Rake task or MetricFu which works with both Flog and Saikuro.

Running Flog against any subset of a project is easy, here I am running it against our client libraries

find ./lib/client/ -name \*.rb | xargs flog -n -m > flog.log

Here some example Flog output when run against our client code.

Total score = 1364.52395469781

Client#send_tests: (64.3)
    14.3: assignment
    13.9: puts
    10.7: branch
    10.5: send
     4.7: send_quit
     3.4: message
     3.4: now
     2.0: create_queue_test_msg
     1.9: create_run_msg
     1.9: test_files
     1.8: dump
     1.7: each
     1.7: report_start
     1.7: length
     1.7: get_tests
     1.7: -
     1.7: open
     1.7: load_file
     1.6: empty?
     1.6: nil?
     1.6: use_cache
     1.6: exists?
ModClient#send_file: (32.0)
    12.4: branch
     5.4: +
     4.3: assignment
     3.9: send
     3.1: puts
     2.9: ==
     2.9: exists?
     2.9: directory?
     1.9: strftime
     1.8: to_s
     1.5: read
     1.5: create_file_msg
     1.4: info
Syncer#sync: (30.8)
    13.2: assignment
     8.6: branch
     3.6: inspect
     3.2: info
     3.0: puts
     2.8: +
     2.6: empty?
     1.7: map
     1.5: now
     1.5: length
     1.4: send_files
     1.3: max
     1.3: >
     1.3: find_changed_files
     1.3: write_sync_time
Syncer#find_changed_files: (26.2)
    15.6: assignment
     8.7: branch
     3.5: <<
     1.8: to_s
     1.7: get_relative_path
     1.7: >
     1.7: mtime
     1.6: exists?
     1.6: ==
     1.5: prune
     1.4: should_be_excluded?
     1.3: get_removed_files
     1.3: find
... and so on ...

Saikuro


Saikuro is another code complexity tool. It seems to give a little less information than some of the others. It does generate nice HTML reports. Like other code complexity tools it can be helpful to discover the most complex parts of your projects for refactoring and to help focus your testing. I liked the way Flog broke things down for me into a bit more detail, but either is a useful tool and I am sure it is a matter of preference depending on what you are looking for.

saikuro screenshot
Saikuro Screenshot

Heckle


Heckle is an interesting tool to do mutation testing of your tests. Heckle currently supports Test:Unit and RSpec, but does have a number of issues. I had to run it on a few different files and methods before I got some useful output that helped me improve my testing. The first problem was it crashing when I passed it entire files (crashing the majority of the time). I then began passing it single methods I was curious about, which still occasionally caused Heckle to get into an infinite loop case. This is a noted problem in Heckle, but -T and providing a timeout should solve that issue. In my case it was actually not an infinite loop timing error, but an error when attempting to rewrite the code, which lead to a continual failure loop that wouldn’t time out. When I found a class and method that Heckle could test I got some good results. I found one badly written test case, and one case that was never tested. Lets run through a simple Heckle example.

#install heckle
dmayer$ sudo gem install heckle

#example of the infinite loop Error Heckle run
heckle Syncer should_be_excluded? --tests test/unit/client/syncer_test.rb -v

Setting timeout at 5 seconds.
Initial tests pass. Let's rumble.

**********************************************************************
*** Syncer#should_be_excluded? loaded with 13 possible mutations
**********************************************************************
...
2 mutations remaining...
Replacing Syncer#should_be_excluded? with:

2 mutations remaining...
Replacing Syncer#should_be_excluded? with:
... loops forever ...

#Heckle run against our Client class and the process method

dmayer$ heckle Client process --tests test/unit/client/client_test.rb

Initial tests pass. Let's rumble.

**********************************************************************
*** Client#process loaded with 9 possible mutations
**********************************************************************

9 mutations remaining...
8 mutations remaining...
7 mutations remaining...
6 mutations remaining...
5 mutations remaining...
4 mutations remaining...
3 mutations remaining...
2 mutations remaining...
1 mutations remaining...

The following mutations didn't cause test failures:

--- original
+++ mutation

def process(command)

case command
when @buffer.Ready then
process_ready
- when @buffer.SetID then
+ when nil then
process_set_id(command)
when @buffer.InitProject then
process_init_project
when @buffer.Result then
process_result(command)
when @buffer.Goodbye then
kill_event_loop
when @buffer.Done then
process_done
when @buffer.Error then
process_error(command)
else
@log.error("client ignoring invalid command #{command}") if @log
end
end

--- original
+++ mutation
def process(command)
case command
when @buffer.Ready then
process_ready
when @buffer.SetID then
process_set_id(command)
when @buffer.InitProject then
process_init_project
when @buffer.Result then
process_result(command)
when @buffer.Goodbye then
kill_event_loop
when @buffer.Done then
process_done
when @buffer.Error then
process_error(command)
else
- @log.error("client ignoring invalid command #{command}") if @log
+ nil if @log
end
end

Heckle Results:

Passed : 0
Failed : 1
Thick Skin: 0

Improve the tests and try again.

#Tests added / changed to improve Heckle results

def test_process_process_loop__random_result
    Client.any_instance.expects(:start_tls).returns(true)
    client = Client.new({})
    client.stubs(:send_data)
    client.log = stub_everything
    client.log.expects(:error).with("client ignoring invalid command this is random")
    client.process("this is random")
  end

  def test_process_process_loop__set_id
    Client.any_instance.expects(:start_tls).returns(true)
    client = Client.new({})
    client.stubs(:send_data)
    client.log = stub_everything
    cmd = DataBuffer.new.create_set_ids_msg("4")
    client.expects(:process_set_id).with(cmd)
    client.process(cmd)
  end

#A final Heckle run, showing successful results

dmayer$ heckle Client process --tests test/unit/client/client_test.rb

Initial tests pass. Let's rumble.

**********************************************************************
*** Client#process loaded with 9 possible mutations
**********************************************************************

9 mutations remaining...
8 mutations remaining...
7 mutations remaining...
6 mutations remaining...
5 mutations remaining...
4 mutations remaining...
3 mutations remaining...
2 mutations remaining...
1 mutations remaining...
No mutants survived. Cool!

Heckle Results:

Passed : 1
Failed : 0
Thick Skin: 0

All heckling was thwarted! YAY!!!

rcov


rcov is a code coverage tool for Ruby. If you are doing testing you should probably be monitoring your coverage with a code coverage tool. I don't know of a better tool for code coverage than rcov. It is simple to use and generates beautiful, easy-to-read HTML charts showing the current coverage broken down by file. An easy way to make you project more stable is to occasionally spend some time increasing the coverage you have on your project. I have always found it a great way to get back into a project if you have been off of it for awhile. You just need to find some weak coverage points and get to work.
Rcov Screenshot
rcov screenshot

ruby-prof


ruby-prof does what every other profiler does, but it is much faster than the one built in to Ruby. It also makes it easy to output the information you are seeking to HTML pages, such as call graphs. If you are just looking for a simple write up to get started with ruby-prof I recommend the previous link. I will talk a little more about the kinds of problems I find and how I have solved them with ruby-prof.

I have used ruby-prof a number of times to isolate the ways to speed up my code. I haven't used it to identify why an entire Rails application is slow (there are better tools I discuss later for that), but if you have a small but highly important piece of code ruby-prof is often the best way to isolate the problem. I used ruby-prof to identified the two slowest lines of code of a spellchecker, which was rewritten to become twice as fast.

Most recently I used it to identify where the code was spending all of its time in a loop for a file syncer. It turns out that for thousands of files each time through the loop we were continually calling Pathname.new(path).relative_path_from(@dir_path) over and over. Putting a small cache around that call essentially eliminated all delays in our file synchronization. Below is a simple example of how a few lines of code can make all the difference in performance and how easily ruby-prof can help you isolate the problem areas and where to spend your time. I think seeing the code that ruby-prof helped isolate, and the changes made to the code might be useful if you are new to profiling and performance work.

changes in our spellchecker / recommender

#OLD Way
 alteration = []
    n.times {|i| LETTERS.each_byte {
        |l| alteration << word[0...i].strip+l.chr+word[i+1..-1].strip } }
 insertion = []
     (n+1).times {|i| LETTERS.each_byte {
        |l| insertion << word[0...i].strip+l.chr+word[i..-1].strip } }
 #NEW Way
    #pre-calculate the word breakups
    word_starts = []
    word_missing_ends = []
    word_ends = []
    (n+1).times do |i|
      word_starts << word[0...i]
      word_missing_ends << word[i+1..-1]
      word_ends << word[i..-1]
    end

 alteration = []
    n.times {|i|
      alteration = alteration.concat LETTERS.collect { |l|
        word_starts[i]+l+word_missing_ends[i] } }
 insertion = []
    (n+1).times {|i|
      insertion = insertion.concat LETTERS.collect { |l|

        word_starts[i]+l+word_ends[i] } }

Changes in our file syncer

#OLD
 path_name = Pathname.new(path).relative_path_from(@dir_path).to_s
 #NEW
 path_name = get_relative_path(path)

  def get_relative_path(path)
    return @path_cache[path] if @path_cache.member?(path)
    retval = Pathname.new(path).relative_path_from(@dir_path).to_s
    @path_cache[path] = retval
    return retval
  end

New Relic


New Relic is a performance monitoring tool for Rails apps. It has a great development mode that will help you track down performance issues before they even become a problem, and live monitoring so that you can find any hiccups that are slowing down the production application. The entire performance monitoring space for Ruby/Rails seems to be heating up. I guess it is easy to see why, when scaling has been such an issue for some Rails apps. Just playing around with New Relic was exciting and fun. I could quickly track down the slowest pages, and our most problematic SQL calls, in this case I was testing New Relic on Seekler (an old project of ours) since I didn't think I would find much interesting on our current Devver site. Seekler had some glaring performance issues and I think if we had New Relic from the beginning we could have avoided many of them. Sounds like I might have a day project involving New Relic and giving Seekler as much of a performance boost as possible. New Relic turned out to be my favorite of the performance monitoring tools. For a much more detailed writeup check out RailsTips New Relic Review.

newrelic screenshot
New Relic screenshot

TuneUp


TuneUp another easy-to-install and use Rails performance monitoring solution. The problem I had with TuneUp was I couldn't get it working on test app for these sorts of things. I tried running Seekler with TuneUp, but had no luck. I found that many people on the message boards seemed to be having various compatibility issues. I looked at the TuneUp screencast and the kind of information that they give you and I feel like this would be equal to New Relic if it works for you. I am emailing back and forth with FiveRuns support who have been very attentive and helpful, so if I get it working I will update this section.

Update: FiveRuns is pretty amazing with their support. I haven't got TuneUp fully working yet, but have made some progress. Some good things to know are that some plugins like safe_erb and output_compression can cause problems with TuneUp. They are aware of the issues, and actively looking into it.

Ruby Run


RubyRun provides live performance monitoring and debugging tools. I hadn't ever heard of this product before I started doing some research while writing this blog article. I am sorry to say but this was the hardest to set up, and gave back less valuable information. I think they need a simple screencast on how to get set up and get useful information back. After getting setup and running I could only get ugly CSV reports that didn't tell me much more than the regular Rails log files. I started reading the RubyRun Manual but it was about as long as Moby Dick and all I wanted was how to view simple easy-to-read reports which is a snap in New Relic and TuneUp. Since the site didn't mention RubyRun providing better data than New Relic or TuneUp which were much more user friendly, I don't think I would recommend RubyRun.

UPDATE: After reading about my difficulties with RubyRun the great folks from Rubysophic got in touch with me. They offered to help me get the tool working and posted a RubyRun quick start guide to their site. I got it working in a snap thanks to an email from their dev and the amazingly simple quick start guide. I still didn't get the same depth of information that I got with New Relic, although RubyRun has a ton of settings so it is likely you can get more depth to the reports. Something worth pointing out is that RubyRun is working on Seekler, which I haven't been able to get TuneUp running on. So if you have been having problems with TuneUp or New Relic, definitely give RubyRun a look. In the end I think the other offerings are slightly more user friendly (less complex settings), and easier to explore the data (link in the feed to both reports, at least when in developer mode). That being said RubyRun offers some great information and options that the others don't and with a bit more UI tuning RubyRun would be at the top of the pack. Thanks to the helpful devs at Rubysophic for helping me to get the most out of RubyRun.

RubyRun screenshot
RubyRun screenshot
RubyRun second screen shot
screenshot of a different RubyRun report

That is it, hope you learned about a new Ruby tool. So get to work, try a new tool, and get to know your code a little better than you did before.

While I was writing this article, people pointed out to me two more tools worth pointing out. I didn't get a chance to try them out or review them, but thought I should point them out. Towlie, helps keep your code dry by finding redundant methods. and finally Source ANalysis (SAN), which is described as, "a Ruby gem for analyzing the contents of source code including comment to script ratios, todo items, declared functions, classes, and much more".

About these ads

Written by DanM

October 3, 2008 at 10:25 am

28 Responses

Subscribe to comments with RSS.

  1. Amazing!
    Thanks man!

    khelll

    October 3, 2008 at 8:37 am

  2. Thanks for this write-up, this was really quite handy. In the back of my mind I always knew that I should take the time to integrate some profiling tools into my app; this was a good kick to get New Relic installed (too many errors right off the bat with TuneUp).

    Cheers!

    Chris

    October 3, 2008 at 8:49 am

  3. Glad you liked the post

    danmayer

    October 3, 2008 at 8:54 am

  4. Awesome, yeah I was blown away with how far the profiling tools have come. They are simply amazing, a no brainer for any new app. It is unfortunate that there seems to be many more problems with TuneUp, I really liked the screen cast, as well as the company FiveRuns, but it seems to be a bit buggier than New Relic.

    danmayer

    October 3, 2008 at 8:55 am

  5. [...] Ruby Tools Roundup [...]

  6. [...] Ruby Tools Roundup [...]

  7. [...] Ruby Tools Roundup [...]

  8. Thanks for the great information!

    Shawn

    October 3, 2008 at 5:43 pm

  9. Hey Dan,

    I sent you an email regarding comments on RubyRun. Being the authors of RubyRun we feel obliged to put some clarification remarks out here to state the facts. Please note that we are not interested in debating with you whether Fiveruns, RubyRun or RPM is better, nor are we trying to change your stand if you have already made up your mind for whatever reason. Just a pure tech exchange.

    1. "…hadn’t ever heard of this product before"?
    - We shipped RubyRun community edition about July 20th 2008. It is at version 0.9 and beta, and it is free for the community. We (Rubysophic) are privately funded.

    2. "… the hardest to set up" ?
    – To install Rubyrun, you use "rubygem install rubyrun".

    – To run your Mongrel or Webrick server with RubyRun just ensure you start the ruby interpreter with -rubyrun at the directory where your application is. TBC

    Joseph Chan

    October 7, 2008 at 5:51 am

  10. 4. "gave back less valuable information"?

    According to the best of our knowledge, one cannot get any of the following currently from the other performance products you mentioned in your article (by the way, most of them are also on the web site):

    - One particular data point you can never get by scraping the logs is the dispatch delay inside an instance of Mongrel or Webrick due to Rails serilaization lock – a hint to increase the size of your cluster as queueing starts.

    - Turing the TRACE property on gives you a request trace with NO code change required. This gives you the entire flow of your transaction of your selected classes/methods

    - Currently tracing also shows all the SQL statements and their response time in context

    - Submitting commands to RubyRun can give you top stack of all running threads to identiy why request hangs

    - Uncaught thread termination will result in a stack trace for further debugging

    - Object map to observe monotonically increasely instances to spot leaks … TBC

    Joseph Chan

    October 7, 2008 at 5:52 am

  11. 5 "… Manual but it was about as long as Moby Dick"

    We take pride in providing all the information regarding the product in a comprehensive way even though RubyRun CE is a freeware. However, we also understand we need to provide a one-page summary of that in case people dont like 'classic', 8-)

    Overall we wish you could have, perhaps, contacted us and spent a little more time before you discount us outright in your blog. Anyhow, at least now that we think we have the chance to let other readers have an alternative view.

    Thanks for mentioning us even though we are not your favorite, and your blog for us to state our story.

    Cheers, Joe.

    joseph Chan

    October 7, 2008 at 5:54 am

  12. Joe,

    I got your email, and will respond to that off the blog.

    While the gem is easy to install, and simple to run the server with, it wasn't obvious how to get data back. I started going through the config files, and the data ruby run created, but I couldn't find much in there. The information was complicated to parse in the format I got it. I do like that RubyRun works for all Ruby projects which is nice, Devver itself isn't a rails app. I am a bit slammed right now, but would be happy to give RubyRun another shot. The biggest help to making ruby run would be a tutorial or screen cast showing how to get at data RubyRun can provide on a simple example project. Show all the configuration changes, where to get the results, and how to modify it for other views.

    I only tried it for a bit, so I can't claim to have much knowledge about RubyRun, my biggest complain was that the others were up and running and giving me feedback I was able to quickly look through and understand. (Even FiveRuns which worked on a smaller sample projects just not on Seekler).

    Glad your working on a interesting tool to help the Ruby community, keep it up.

    danmayer

    October 7, 2008 at 6:33 am

  13. Dan,

    Thanks for creating this useful Ruby tools overview. It's a great resource for the Ruby community.

    At FiveRuns, we are still stumped that TuneUp isn't working with your application yet. We have improved the Rails plugin in recent months to address most of the issues discussed in the forums. You're one of the very few customers that can't use TuneUp.

    We are still working to reproduce your exact error condition. So, in the near future, please expect a TuneUp update with some internal changes that will hopefully fix the problem for you.

    Oliver from FiveRuns

    Oliver Schmelzle

    October 10, 2008 at 6:42 am

  14. [...] Ruby Tools Roundup – The Devver Blog [...]

  15. Amazing post! Really nice contribution to the community

    Gabriel Miretti

    October 16, 2008 at 9:05 am

  16. Very usefull artice! Thanks for sharing!

    sebastian

    October 19, 2008 at 2:45 am

  17. Good stuff, thanks! You might also want to look at reek: http://rubyforge.org/projects/reek

    Kevin Rutherford

    November 14, 2008 at 3:52 am

  18. [...] are used by a lot of the Ruby code inspection tools to understand your ugly Ruby [...]

  19. [...] Ruby Tools Roundup [...]

  20. [...] in under 50 lines with Sinatra and iUIEmili Parreño » Blog Archive » Enlaces (1) on Ruby Tools Rounduplinks for 2008-11-26 « Brent Sordyl’s Blog on Building a iPhone web app in under 50 [...]

  21. Miks…

    I stumbled across this post in google and it helped me fill in some blanks. I took alook at some of your other content and you have a good site, keep the posts coming….

    Miks

    March 3, 2009 at 10:24 am

  22. Good job~I like this post indeed.

    Robert Motherwell

    May 14, 2009 at 8:08 pm

  23. I just released Excellent which implements several checks on Ruby code. It combines roodi, reek and flog and also adds some Rails specific checks:

    <code>gem sources -a http://gems.github.com
    sudo gem install simplabs-excellent</code>

    May be helpful…

    marcoow

    May 21, 2009 at 5:20 am

  24. Nice.Very clear and details.Learn a lot here.

    mathew

    June 15, 2009 at 3:47 pm

  25. [...] can try these metrics out on their projects. I think metric_fu is awesome, and my interest in Ruby tools is part of what drove us to build Caliper, which is really the easiest way try out metrics for your [...]

  26. [...] can try these metrics out on their projects. I think metric_fu is awesome, and my interest in Ruby tools is part of what drove us to build Caliper, which is really the easiest way try out metrics for your [...]

  27. [...] under: Development, Hacking, Misc, Ruby, Tools | I have heard about new Ruby tools since I did my Ruby Tools Roundup. I am always interested in tools that can help improve our code, so I had to check some of them [...]

  28. Nice, Very helpful :)

    Loganathan

    June 15, 2012 at 4:21 pm


Comments are closed.

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: