The Devver Blog

A Boulder startup improving the way developers work.

Archive for December 2008

Boulder CTO December Lunch with Tim Wolters

The Boulder CTO Lunch meets once a month with a guest speaker and covers topics and questions that startup CTOs should find interesting. This month, the group had Tim Wolters from Collective Intellect come lead the discussion. Tim is a serial entrepreneur currently working on using artificial intelligence and semantic analysis to extract knowledge from unstructured text found in social media. Collective Intellect’s customers use this analysis to inform and measure the effectiveness of their PR and marketing strategy.

Tim is considering working on a book, a startup survival guide for CTOs. Some of his ideas for the book helped lead our discussion during our meeting. I will try to present my notes under topic headings that Tim mentioned, but since this was a open free formed discussion, I am sure I couldn’t capture everything and not all my notes are completely accurate.

The Idea
People should keep a journal of ideas. Tim keeps a journal which he updates, tags, and adds ideas. On any idea, keep track of what is near term, what resources are needed, what is the cost, and what does the related market look like. (I highly recommend this! Ben and I keep a wiki, which has grown to be an incredibly useful resource and was the initial starting point for our last two companies)

Ideas should have an “Aha!” factor that makes you wonder why someone else isn’t already doing it (or some emotional appeal that makes lives better).

During the first few years of a startup you can’t work on all the ideas that come to mind, that is why it is best to keep a journal, just add little notes to the idea to keep them in the back burner.

Talk to others about ideas and perhaps have a group move on an idea and lay the groundwork while leading as an adviser.

Don’t be worried about people taking ideas. After starting a few companies you know how hard it is to really bring something to market.

What about brainstorming for ideas with a group?
Brainstorming groups have never worked for Tim, it just hasn’t worked out. If you have the right people around the table (people that can make things happen), it could work, but Tim hasn’t seen it.

Ideas depend a lot on timing in the marketplace. If the market is moving slow you can slowly look at an idea. If the market it really moving fast you need to spin it up quick and get a lot of people working on it to really make a move on the idea.

Look over your ideas once in awhile and see what still really interest you.

The Role
As a CTO, you paint a landscape of the product and market.

There are two kinds of CTOs: tactical and visionary. Tactical CTOs are internally focused, manages the team, makes the day-to-day tactics so the product gets out there. The visionary CTO sees where the product could go in the market place, signs early deals and customers, looks for features that lead towards or away from markets/competitors/partnerships. The visionary isn’t working on architecture but the market landscape, what partners will benefit the product or get it out sooner.

CTO should be thinking about things such as the three hardest problems that the company faces, so they know what will also be affecting their competitors.

People who liked architectural purity but learned it isn’t as import at winning at the business end up making great CTOs.

CTOs need to stay involved with customers to make decisions about the project innovation and development. Stay active on sales calls, talk with sales people, read all the RFPs.

Becoming the CTO vs VP of engineering?
Are you good at managing or not? VP of engineering is a managing role. If not, divide off the management as soon as possible (in his case that wasn’t possible until the company was about 20 people).

Good sales people leverage a CTO as a company evangelist. If you are a CTO you have to be comfortable with presenting and publicity. You will be at conferences, sales calls, giving presentations, and fund raising. If you aren’t comfortable with these things, get comfortable with it.

time spent:

  • 10% guiding research
  • 30% Sales
  • 30% Partnerships
  • 30% Biz Dev Dealings

Reputation
After some startups, successes, and expanding your network things like getting a team, funding, and getting a startup off the ground are much easier the next time.

It will take 3 to 5 times longer than you think to get a project going if you are an unknown entrepreneur with no reputation.

Don’t solve the big unsolvable problems first, the first time start with smaller problems and develop a reputation while solving them. Angels and VCs aren’t funding research efforts, don’t just chase after big impossible goals.

After a company is bought, it makes sense to make the purchaser successful. It builds on your reputation.

Become a big fish in a small pond and then move to a bigger pond.

Putting together the team
The ideal size for an engineering team is 6-8 people, bigger teams have difficulties maintaining the right amount of communication.

For hiring, Tim personally sits down with the key hires, and if it is research he does interviews with applicants as well.

The Traps and Pitfalls of Startup Companies
3 things that companies get stuck on that can kill the company.

  • Problem with getting over enamored with their original idea, startups must be able to adapt
  • Getting enamored with the research technology, for technology’s sake
  • Getting emotionally tied to architecturally purity. Working on layers of abstraction on abstraction to avoid some possible future problem.

Other things that kill companies (which are kind of like a marriage)

  • Not the right chemistry
  • Bad culture or losing company culture
  • Employees need some sense of allegiance. If they don’t have it cut them immediately
  • Lacks a culture of adaptability
  • Not thinking about how to quickly get to the market and solve problems

Continual code death march. Sometimes companies go on code marches to get something to the marketplace. This can’t be done many nights or it will start taking a toll on other aspects of your life. Strive for balance.

During a startup, you continually are hitting false summits, you think that if you could just get that contact, solve that roadblock, pass this milestone, or make this key hire then everything will fall into place. While these are important as milestones and you should celebrate them you are not done. Or rather, it typically doesn’t get any easier. What it does is takes more risk out allowing you to go solve bigger/other problems.

When founders or others in a company argue, which they need to do sometimes, don’t do it in front of everyone. Discuss disputes offline, reach agreement and present a unified front to the company.


Thanks so much to Tim for sharing some of his thoughts with our group. I will leave you with a final question and quote. Someone once asked why Tim likes to start companies?
“I like to pick where I work and who I work with.”

Written by DanM

December 11, 2008 at 9:50 am

Installing and running git-svn on Mac OSX 10.4 Tiger

I am shocked at how much time it took me to get git-svn working on my mac. I use MacPorts, which works well most of the time. Sometimes it has problems which makes me really wish for apt-get on OS X. apt-get normally has worked much nicer for me, but can have its issues too. I even occasionally wish for Windows and a simple install.exe which works 95% of the time out of the box. Really I wish Apple would throw some engineer support to MacPorts and make the service rock solid.

I have had git installed and working for awhile, but preparing to switch our main project from Subversion (svn) to git, I thought I should start using git-svn. It seemed smart to use git-svn for awhile to get used to git, before a full switch so I could fall back on svn in a crunch. I decided to start using git-svn, but the first run of the git-svn command caused this error, and I had no idea how much of my night was about to be wasted…

Can't locate SVN/Core.pm in @INC

Searching led to a couple of webpages, but the most useful was getting git to work on OS X Tiger. It had a quick fix that might work or the long route fix. For some lucky people it is just a path problem. I checked if that was the case for me, by the following command

PATH=/opt/local/bin:$PATH; git svn

unfortunately for me I got the same error, OK I need to reinstall SVN with additional bindings…

> sudo port uninstall -f subversion-perlbindings
> sudo port install -f subversion-perlbindings

leading to this error:

--->  Building serf with target all
Error: Target org.macports.build returned: shell command " cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_www_serf/work/serf-0.2.0" && make all " returned error 2
Command output: /opt/local/share/apr-1/build/libtool --silent --mode=compile /usr/bin/gcc-4.0 -O2 -I/opt/local/include -DDARWIN -DSIGPROCMASK_SETS_THREAD_MASK -no-cpp-precomp -I. -I/opt/local/include/apr-1 -I/opt/local/include/apr-1  -c -o buckets/aggregate_buckets.lo buckets/aggregate_buckets.c && touch buckets/aggregate_buckets.lo
libtool: compile: unable to infer tagged configuration
libtool: compile: specify a tag with `--tag'
make: *** [buckets/aggregate_buckets.lo] Error 1

I spent some time searching and eventually I find the solution to the serf error. I couldn’t read the blog because it wasn’t in English, but I could read enough to solve my MacPorts serf install problem. I followed these few lines from the blog

cd /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_www_serf/work/serf-0.2.0
$ sudo ./configure --prefix=/opt/local --with-apr=/opt/local --with-apr-util=/opt/local
$ sudo make all
$ sudo port install serf

Awesome, I have serf. Now what is next? Back to building svn with perl bindings, that works. Now, let’s build git again since svn with perl bindings is finally installed.

sudo port install git-core +svn

Which fails because of p5-svn-simple

dyld: lazy symbol binding failed: Symbol not found: _Perl_Gthr_key_ptr
Referenced from: /usr/local/lib/libsvn_swig_perl-1.0.dylib
Expected in: flat namespace
dyld: Symbol not found: _Perl_Gthr_key_ptr
Referenced from: /usr/local/lib/libsvn_swig_perl-1.0.dylib
Expected in: flat namespace
Error: Status 1 encountered during processing.

OK, I need to get p5-svn-simple working. Searching leads to this thread MacPort errors related to git. Here you will find the amazingly useful comment by Orestis:

“As mentioned move your libsvn_swig_perl* out of /usr/local/lib AND out of /usr/lib into temporary folders.

Uninstall and reinstall subversion-perlbindings

Install p5-svn-simple (and git-core +svn which is what lead me here)

Move the libsvn_swig_perl files back in /usr/lib and /usr/local/lib (or else git svn won’t work).

> cd /usr/local
> mv ./lib/libsvn_swig_perl* ./bak/
> sudo port install p5-svn-simple

Sweet that works now

> sudo port install git-core +svn
> cd /usr/local
> mv ./bak/libsvn_swig_perl* ./lib/

Finally I try to run git-svn, only to see the same ERROR I had from the very beginning! I am about to lose it but decide that I should try the quick fix again to see if it is the path issue…

PATH=/opt/local/bin:$PATH; git svn

It works! Alright now it is just a path problem. So I open up my .bash_profile, and notice I already have that path included

# Setting the path for MacPorts.
export PATH=/opt/local/bin:/opt/local/sbin:/Applications/MzScheme\ v352/bin:$PATH

But I also have an additional path added from when I originally built git from source, and it looks like I was running my old broken version of git-svn. So I just had to remove this one line from my .bash_profile

export PATH=~/projects/git-1.5.6.1:$PATH

and hours later and with a ton of frustration I have a fully functioning git-svn.

Now that it is working, you can move on to learning git-svn in 5 minutes.

Written by DanM

December 9, 2008 at 11:16 am

Revisiting additional Ruby Tools

I have heard about new Ruby tools since I did my Ruby Tools Roundup. I am always interested in tools that can help improve our code, so I had to check some of them out. Similar to my last tools post, I will be trying out a tool and writing my general impressions along with the basic usage.

reek


I have to start with reek, since it has been the most requested and searched on our site since I originally wrote about tools. reek will help identify code smells, allowing you to fix up your code. Instead of looking at cyclomatic complexity or other metrics, reek looks at patterns to warn you about bad code. Reek currently detects a few code smells (Long Method, Large Class, Feature Envy, Uncommunicative Name, Long Parameter List, Utility Function, Nested Iterators, Control Couple, Duplication) but more are on the way.

I think this project is useful but would need to be more customized before a nightly run would yield very useful results. The biggest problem I have is the signal to noise ratio seemed pretty high. Reek was warning me about “long methods” that were only 7 statements long, which just isn’t something I am concerned about. The warnings on duplicate methods calls can be useful, after running reek on a few files I found a couple places where duplicate method calls were wasting time. Many of the other smells are interesting like ‘Feature Envy’, and ‘Utility Function’. I will need to use reek more before I know if these smells are good indicators or often false positives.

Below reek finds a utility function next_tick which is definitely a helper function that actually exists in two of our files, which probably should be moved into a helper mixin.

def next_tick
    if(EM.reactor_running?)
      EM.next_tick do
        yield
      end
    else
      yield
    end
end

I am really looking forward to see how the tool progresses. If the project allows for a simple config customization to change the thresholds as well as ignore some files/smells, this could become a very useful tool to help keep a team maintain a high expectation of code quality. It would be useful to get nightly reports about any code that might not meet expectations, so a quick group code review could decide if it is an exception (which can be quickly added to the config) or if the code should be refactored and cleaned up.

dmayer$ sudo gem install reek
dmayer$ reek ./lib/client/client.rb
[Utility Function] Client#next_tick doesn't depend on instance state
[Long Method] Client#process_done has approx 7 statements
[Duplication] Client#process_ready calls @buffer.create_reload_msg more than once
[Long Method] Client#process_ready has approx 10 statements
[Duplication] Client#report_system_message calls result.msg more than once
[Feature Envy] Client#report_system_message refers to result more than self
[Duplication] Client#send_tests calls Time.now more than once
[Long Method] Client#send_tests has approx 24 statements
[Feature Envy] Client#send_tests refers to tests more than self
#check a whole directory
dmayer$ reek ./lib/client/*

Towelie


Towelie helps discover duplication in Ruby code, it will help keep your code DRY. It doesn’t have a nice interface at the moment and it is pretty young code. That being said, it can still be a really useful tool to help guide refactoring and code cleanup.

~/projects dmayer$ git clone git://github.com/gilesbowkett/towelie.git
dmayer$ cd ~/projects/devver/
dmayer$ irb -r ~/projects/towelie/lib/towelie.rb
irb(main):001:0> @t = Towelie.new
=> #, @model=#>
irb(main):002:0> @t.parse "lib/client"
(string):24: warning: useless use of a variable in void context
=> nil
irb(main):003:0> puts @t.duplicates
found in:
lib/client/test_unit_reporter.rb
lib/client/rspec_reporter.rb

def nl
report_nl
end

... 2 more dupes in the reporters ...

found in:
lib/client/test_unit_reporter.rb
lib/client/rspec_reporter.rb

def report(str)
print(str.to_s)
end

found in:
lib/client/sync_client.rb
lib/client/rev_sync_client.rb
lib/client/rev_client.rb
lib/client/client.rb

def quit
send(@buffer.create_quit_msg)
end

found in:
lib/client/sync_client.rb
lib/client/rev_sync_client.rb
lib/client/rev_client.rb
lib/client/client.rb

def send_quit
send(@buffer.create_quit_msg)
end

=> nil
irb(main):004:0>

There are currently many duplications because we are maintaining two clients while deciding what route to eventually take. We have also moved a lot of our shared client code into a mixin, and Towelie finds some methods that really should be moved there as well such as the methods “quit” and “send_quit”, which is currently duped in 4 files. Towelie also points to the fact that we should refactor our reporters because they both duplicate code.

I have always been annoyed with copied and pasted functions accidentally working its way in code, this could be a useful nightly run to keep a team DRY. Sometimes two team members implement the same functionality without even knowing a solution already exists in the code base. If you want to go a bit more in depth, check out Giles Bowkett’s (creator of Towelie) How to use Towelie

Flay


Flay is another great tool by Ryan Davis who also works on Heckle and Flog which I covered in the past. Flay, like Towelie, helps keep your code DRY, it detects exact and similar code throughout a project. It seems to be more powerful than Towelie, as seen in this Towelie and Flay comparison. My biggest complaint is the current release has some pretty basic output that you see below. The output I got from Towelie was immediately more recognizable and useful, while Flay currently requires you to dig in a bit deeper on your own into its suggestions. An improvement is already being worked on and a verbose output mode should be in the release soon. Once better output is included I think Flay will be immediately useful out of the box even with small amounts of developer effort.

I like that Flay has weight system, which should make it easy to set some threshold to ignore, high level weights are more likely to be worth your time and attention. One piece of code Flay tagged with a low weight was code that rescued and logged different errors thrown, which while similar actually served a purpose.

rescue Errno::EISDIR => ed
      @stderr.puts "Error: #{ed.message}" if @stderr
      @stderr.puts "You can't pass a directory to devver only test files. Quitting." if @stderr
      send_quit
    rescue LoadError => le
      @stderr.puts "Error: #{le.message}" if @stderr
      @stderr.puts "Not all of the files can be found. Quitting." if @stderr
      send_quit
    rescue SyntaxError, NameError => se
      @stderr.puts "Error: #{se.message}" if @stderr
      @stderr.puts "This file doesn't appear to be a valid Ruby file. Quitting." if @stderr
      send_quit
end

Digging into the Flay results turned up some duplicate code that Towelie had missed. Since Towelie also caught a method that was duped in 4 client files that Flay missed (I was expecting Towelie’s results to be a subset of what Flay found), perhaps there is room for both of the tools and learning to work with both a little bit is worth the time. After a little bit of work perhaps one of the projects will become a clearly better option. Until then I will be following both of these projects.

sudo gem install flay
dmayer$ flay lib/client/*.rb
Processing lib/client/client.rb...
Processing lib/client/mod_client.rb...
...
Processing lib/client/syncer.rb...

Matches found in :defn (mass = 84)
lib/client/mod_client.rb:86
lib/client/mod_rev_client.rb:124

Matches found in :block (mass = 57)
lib/client/client.rb:201
lib/client/client.rb:205
lib/client/client.rb:209

... 6 more results ...

Matches found in :if (mass = 34)
lib/client/mod_client.rb:63
lib/client/mod_rev_client.rb:111

Matches found in :defn (mass = 32)
lib/client/mod_rev_client.rb:36
lib/client/mod_rev_client.rb:50

Conclusions


That should cover it for this Ruby tools post, but I am really enjoying checking out the tools showing up in the Ruby scene. So as always let me know if I missed something, or if there is a tool you would like to see a full write up on. After some of the tools mature a little bit I will have to revisit a few of the tools which are currently in the early stages. I hope the Ruby tools scene keeps as active as it has been lately because there are some interesting projects being worked on.

honorable mentions (things I didn’t think really needed a full write up)


  • metric-fu a great gem to give quick access to a bunch of tools and metrics about your code (RCov, Saikuro, Flog, SCM Churn, and Rails Stats)
  • CruiseControl.rb when you start using all of these tools, continuous integration starts to become more important (or doing nightly runs). CruiseControl.rb is dead simple continuous integration.
  • Simian another code duplication tool, which is mentioned in 3 tools for drying your Ruby code (free for OSS, $99 for a license)
  • Ruby Tidy a tool for cleaning up HTML (I haven’t used this in Ruby, but loved the Java version in my Java days)
  • Watir is an open-source library for automating web browsers. It allows you to write tests that are easy to read and maintain. It is simple and flexible.
  • Autotest, if you haven’t heard of autotest, check it out, continuously run your tests every time you save a file in your project.
  • Rufus a tool that checks if code you are about to load is safe. Allows you to look for custom patterns that you don’t want to run.
  • I wrote about a couple benchmarking tools last time and here is a great article / tutorial on Ruby benchmarking

Written by DanM

December 3, 2008 at 10:01 am

Devver.net is hiring!

We are looking for a top-notch Ruby hacker who wants to help us build the next generation of developer tools.

At Devver.net, you’ll be solving new and interesting problems on a daily basis. You’ll help us improve our architecture and build new features.

If you have a passion for Ruby, developer tools, and highly distributed systems, please read more about the job and contact us at jobs@devver.net

Written by Ben

December 2, 2008 at 10:35 am

Posted in Devver