Caliper: Next Steps
We released the first version of Caliper just under two months ago. Since then, we’ve fixed bugs, made improvements, and rolled out new features. The feedback we’ve gotten from the community has been encouraging and your suggestions have helped up improve Caliper each week.
Initially, we simply wanted to take an existing tool (namely, metric_fu) and make it dead simple to set up, so that all open-source projects could get access to metric data. However, our larger goal is to improve the usability, features, and data presentation in Caliper so it becomes an indispensable part of your regular Ruby development process.
To that end, I’m happy to announce that support for private projects is on the road map. We’re still working out the details, but we plan to release this feature within the next thirty days.
At the same time, we’ll work with users to figure out how we can make metrics useful enough to be a regular part of your process (by “regular”, we mean that we hope you use metrics at least once a week).
We recently put out a short survey (results here) to try to assess what features are most important for our users. By looking at the results (somewhat limited, likely due to overly wordy questions on my part) and based on some conversations I had at RubyConf, we see a few possible scenarios for where to take Caliper next.
In this scenario, you’d primarily use Caliper to jump into an existing code base, in order to help you better understand where the problematic areas are. This might be used by consultants for a project-level code review, for understanding how to start contributing to an open-source project, or even to periodically do a survey of problems of your own project. The primary features for this scenario would be our Hot Spots feature (which list the worst methods, classes, and files in the project). Important new features include a display that shows the source code with the problems displayed alongside the source.
In this scenario, the primary usage for Caliper is to set and maintain certain metrics thresholds within your code base. You’d set levels for the various tools and then get alerts when those thresholds are reached. Key features would include an ability to customize thresholds for each tool, a dashboard that displays offending classes and methods, and customizable notifications.
In this scenario, a single developer uses Caliper to assist them in refactoring up his/her code, perhaps after reviewing a commit that will be pushed, or because a specific section of code is known to be overly complex. Caliper would both find the biggest problems and provide feedback on how refactorings change the metrics. Key features would include an improved commit-based view of metric changes as well as the ability to push code directly to Caliper before merge that code into a shared ‘master’ branch.
Do any of these scenarios resonate with you? Would you prefer one over another? Is there another scenario that would be more useful? If you are currently using metrics in your project, how do you use them and what features do you want? If you don’t currently use metrics, what features would make you change your mind? Please make your voice heard in the comments, so we can make Caliper a valuable addition to your toolbox.