Skip to content

The only software metric you need…

This is a continuation from my previous post on ‘Why measuring productivity is bad for business‘ inspired by Dan North’s Accelerated Agile course.

Note: This post has been updated based on feedback in the comments. I was confusing lead and cycle time. Thanks Paul!

If measuring productivity is bad for software then what on earth should we measure? Initially I argued with Dan that it must include something about value delivery. His take is: that is a problem for the whole business, but if you want to look at software development in isolation then the key metric has to be about how quickly the team deliver stuff, but we know we don’t actually care how productive they were, so what does it mean?

It’s simple – lead time – how long does it take from when you identify your customer needs something to when they get it. That’s it!

If you team can deliver what the customer wants as close as possible to when they want it you will win.

So how do you measure this?

Here are 3 simple ways.

1. If you use a ticketing system like Jira, lead time is simply the time the issue was logged up until the time it was resolved. In my experience as soon as you do this you will realise half the stuff in your Jira system is out of date, and so the data is no good. Try approach number 2.

2. Whenever you add an item to the product backlog add the date (if you are using physical card, just write it on one corner). Whenever you ship a feature add the date. Now just subtract the two to get lead time. Of course that might not work for everything on your backlog today. Try approach 3 for that.

3. If you have a ton of stuff on the backlog you have no idea when it was added, and generally new stuff is not added, then try looking  at just the cycle time. When a story is started in a sprint note the date, and when a story is release to customers note the date. Subtract the two to get your story cycle time. Remember this is just a piece of the lead time because it excludes the wait time it sat on the backlog for, but it’s a good place to start. See this link for a great explanation.

How can it be gamed ?

If you know anything about metrics, you will know they can and will be gamed as soon as you start measuring them. So how can you game lead or cycle time?

The two patterns I’ve seen are:

a) It’s not in scope. Team members can start to nitpick around scope, if their focus is on getting things done as soon as possible. The PO wants the text worded differently and the team request a new story for that because “it’s not in scope”. While in general this is not a bad idea, you probably want to avoid extremes. As a counter metric you could track how many new stories arise because of ‘scope creep’ and how big they are. If you’ve got a lot of very small stories being added every sprint, cycle time is probably being gamed.

b) High failure rate. In an effort to finish things quicker teams can unconsciously drop quality. We all know this, and so of course it can happen if teams are chasing a short cycle time. The best way to balance this is with a failure load metric that tells you how many things come back because of errors.

c) Some people think that to game this teams will want to break stories down as small as possible, particularly if you are only measuring cycle time in the sprint (option 3 above). However in my experience that’s a good thing, I have rarely seen a team break down stories too small. If you find this happening and the team have >20 stories in a sprint, maybe try measuring the full lead time from customer request to delivery.

How can we reduce our lead time?

The simplest possible way (which can be mathematically proven using queueing theory or demonstrated with the airplane game), is to reduce work in progress. Stop starting things and start finishing things. Here are a few ways you can do that to improve lead time:

  1. For Product Owners this means culling your backlog. Let’s be honest. Those features that have been sitting in your backlog for 2 years are never going to be important enough to get done. Be bold and delete them. Think about what your ideal lead time is for your product, and then delete anything older than that in your backlog. i.e. if you’d like a lead time of 3 months, take everything older than 3 months and delete it from your backlog. Trust me, if it is important it WILL come back.
  2. For teams this usually means working on only one or two stories as a time, and keeping stories small – it helps get them ‘done’ sooner. This means collaborating and having multiple team members work on the same story. If you think that’s impossible, you might need to look at your version control strategy or your system architecture and find ways you can make this possible.
  3. For releasing this means smaller more frequent releases. I guarantee you have a process to get a hot fix to a client if they encounter a sever bug in production. Why can’t that process be the way you release all software? The worst wastage in lead time is having a feature done and not making it available to the client. 

Good Luck!

kanbanbook— Update October 2016

If you enjoyed this blog post, you will love our book Kanban Workbook: A Practical Guide to using Kanban.

The book will guide you through implementing your own Kanban board and all the principles of Kanban, including metrics like lead and cycle time, in your own environment.

 

Tags:

6 thoughts on “The only software metric you need…”

  1. Paul Klipp

    The idea is nice, but I think you’ve got an unusual definition of cycle time which could create confusion. Traditionally, cycle time is the time spent in development and lead time includes cycle time plus any wait time on both ends.

    I find it best to measure lead time from the time the client pulls a task from the backlog to the “Next” or “To Do” or “Sprint backlog” column (not from the product backlog – I think a product backlog is just an idea list, much of which won’t ever get done) until the client has used the feature to solve the problem that demanded the feature in the first place. I measure cycle time from the time the first team member pulls the task to work on it until it’s ready to deliver to the customer.

    I’d also take issue with the idea that what the customer requests is what they need. Too often, that’s not true at all. That’s why I like to validate every feature before I call it done. If it doesn’t solve the problem, then it should be reworked or removed.

    1. Karen Greaves

      Thanks Paul for the great comments. I do get mixed up with lead and cycle Time because I’ve seen it used both ways. Do you have a good source I can link to with a clear definition? Would like to add that and then fix up how I use it in the post.
      Also love the idea of validation afterwards, because you are right not everyone asks for what they need.

      1. Paul Klipp

        One irrefutable, if not quite fully relevant, source is Toyota, because the terms come to us in the software industry from TPS.

        From a production standpoint, you can find the definitions here: https://www.lean.uky.edu/reference/terminology/

        David Anderson does a very poor job of addressing this question in his Kanban book, but Stefan Roock gives an excellent definition here: http://stefanroock.wordpress.com/2010/03/02/kanban-definition-of-lead-time-and-cycle-time/

        The big question is how you define a request and how you define starting work on an item. As I’ve said, I don’t consider a backlog to be a request for two reasons. 1) It’s not useful, as the queuing time for the last backlog item is huge, relative to the size of the project and 2) it’s probably bigger than huge; it’s infinite.

        Ultimately, you have to use what makes sense in your context and stick with it, but the lead time should always be greater than the cycle time, but a small amount if you have a very small input queue and continuous deployment, and by a large amount if you need to maintain a large input queue and have long release cycles. As long as the definition is consistent, the metrics can be compared over time.

        1. Karen Greaves

          Thank Paul, I will update the blog post soon to correct lean and cycle time. Really appreciate your input.

          As for a long queuing time in the backlog, part of what Dan is talking about is not having a huge backlog where stuff sits indefinitely. i.e. when you identify the need (however you do that), build it as soon as possible and validate it in production, or agree not to do it, don’t just add it to a queue. I often find backlogs with items in them that have been there for more than a year and are no closer to the top, I usually advise people to delete them.

Comments are closed.