Matt Bernier | 06 October 2017
Improve the efficiency of your software development team with the RICE framework. Learn more at the upcoming APIStrat conference in Portland, Oregon.
The Developer Experience team at SendGrid is a small, but mighty force of two. We attempt to tackle every problem that we can get our hands on. This often means that some items get left behind. At the outset, we surveyed everything that was going on in our open source libraries and we quickly realized that we needed to find a way to prioritize what we were going to work on. Luckily, our team lives, organizationally, on the Product Management team, and we had just received a gentle nudge and training on the RICE prioritization framework.
On our company blog, I wrote an article about how employing this framework, using a spreadsheet, helped us double our velocity as a team within the first sprint. Our development velocity doubled because the most impactful things for the time spent are not always the biggest things, but the biggest things tend to attract the most attention due to their size.
What is RICE?
RICE as an acronym stands for Reach x Impact x Confidence, all divided by Effort. This calculation allows you to get a score that weighs the following. Some of the definitions we use are a slight departure from Intercom’s original version, but this has been very effective for us!
The calculation:
Reach * Impact * Confidence
————————————–
Effort
This gives us a score for every item in our list. Then, we sort our list in descending order by score. We realized, once we had a sorted list, that we accidentally made a Kanban backlog. We worked from the top of the list, keeping work in progress (WIP) to as much of a minimum as possible. WIP can be tough with open source, because we often have 20-30 issues waiting for a community member response. These items sit at the top of our backlog, and we look into them at the start of every day in the hope that we can clear them out of our WIP category.
Lessons Learned
Reach – The number of customers this will affect
One thing we learned about using RICE is making sure that we use consistent numbers for each of the variables in the calculation. It was very tempting for us, an email company, to use the “number of emails” sent as the Reach parameter. This worked until we started trying to evaluate tasks that didn’t have anything to do with our v3/mail/send endpoint. We eventually settled on number of customers using this library for this purpose”, calculating API user count and mail user count for the Reach.
Impact – A measure of the effect completing this project will have
It is easy to assume that every single item is a high or massive priority. It looks nice, gives you an ego boost, and totally messes up everything in your ranking system. Be honest with yourself about what is on your list. If things don’t really seem to be in the right place in your list (more on this below) then look at impact, because it’s probably artificially high, especially in context of the items around it in the list.
Confidence – How confident are we that we can sit down and complete this task today
We use a text-based selection from the list: None, Minimal, Low, Medium, High, and “with my eyes closed”. These each translate into specific numbers for the calculation.
Effort – The number of story points will this take to complete
We use story points because this approach allows us to figure out the calendar length of a task rather than the aggregate of specific amounts of time spent on the task. To be more specific, this is the difference between “It’s going to take 3 hours” and “It will only take 3 hours of work, but it won’t be finished until Friday.” This is an easy trap to fall into, because new projects are exciting and we want to jump in and knock them out. That doesn’t mean that we can actually get a project that “will only take a couple days” done within the same month we started it. Life happens, your velocity calculation accounts for that — especially if you use an average over the last year (get out yer agile pitchforks!).
Letting the backlog win
We have learned the hard way with projects we really want to work on right now, that they are not always the right project at the moment. It is important to have confidence in the calculation and take the assumption that it is correct. That is, until you are looking at the list and realize that “Hey, item 15 really should be item 5. What’s going on here?” Look at your list in context of the other items, does the order feel correct? If not, why not?
We ended up using RICE as the baseline calculation for everything we do, but it is not the end-all. We added in calculation modifiers for company priority, due dates, and status — because something the executives have on the company roadmap, that has to be done in Q3, should be worked on right away. And, because we are using Kanban, the status of an item is important. Once you start something, it should stay at the top of the list until it is done, or you decide it is no longer necessary to complete it. Getting WIP completed, rather than backed up, is a good way to see the impact of your work and get a sense of accomplishment for yourself and your development team.
Matt Bernier is the Developer Experience Product Manager at SendGrid, where he spends most of his time digging into customer feedback in order to provide a World Class Experience for Developers with SendGrid.
Learn more in Matt Bernier’s talk — How We Doubled the Velocity of Our Developer Experience Team — at the APIStrat conference coming up Oct. 31 to Nov. 2 in Portland, Oregon.