Workshop: Estimation Techniques

This week I ran an estimation techniques workshop to help my teams understand different ways of estimating. I’ve run it once before at my previous company and people found it useful. This time Sam and I prepared together, and she ran the same workshop with her teams. We agreed to blog about the results to see how it compared.

Goal

So why run the workshop? Currently my teams all use planning poker, however I’ve noticed 2 things starting to happen. I think these are fairly common in teams using planning poker.

1. Sometimes they get very caught up in the actual number. Forgetting it’s just an estimate and wrong anyway.
2. Sometimes people get emotionally invested in being ‘right’. People start ‘defending’ their estimates, or giving in to the majority without discussion.

I like to remind people that what we care about is getting a shared understanding and consensus, not necessarily worrying too much about the actual number. Remembering that this is just a way to plan, and help us know where we are against that plan. When we have large numbers of stories, and work with an average velocity, it turns out that if one particular story is a 3 or a 5 doesn’t matter all that much.

So what I wanted the teams to see is that you can get estimates about 80% right with 20% of the effort, and it’s not worth the other 80% of the time. I also wanted them to understand that we really are better at relative estimating than at absolute estimating. And finally, I wanted them to try some different techniques to see if they could learn something that might work better for them in the future.

The workshop

The format of the workshop was 4 exercises, each time boxed to 10 minutes. For each exercise the teams had to estimate items using a different technique. Each team got a set of cards with the items written on the cards. Each team got the same items so we could compare between teams at the end. When each team had finished estimating I wrote up the results of each team and we had a discussion about that technique. I also shared the ‘correct’ answers with them, because I know that people always want to know how right they were. At the end we had a general discussion about all the techniques and decided where we’d go from here.

Absolute Estimates

For exercise 1 we did absolute estimating, and the items to estimate were dogs. Teams had to estimate the weight of the dogs in kilograms.

Here are the results:

Exercise 1: Absolute Estimates					TIME	3:18
	Group 1	Group 2	Group 3	Group 4	Group 5	Correct
Chihuahua	1	1	1	1	7.5	3
Great Dane	75	80	70	60	75	90
Staffordshire Bull Terrier	16	10	50	25	40	17
Appalachian Mountain Dog	45	60	70	35	30	0
Border Collie	20	20	45	20	35	34
American Cocker Spaniel	15	12	35	30	50	13

Although to my surprise this turned out to be the fastest technique at 3minute 18 seconds, when I asked for feedback people commented that they were “pure guesses“, and that looking at the results they were “quite bad at it“. At the end of the discussion I pointed at that there is no such thing as an Appalachian Mountain Dog, and yet everyone thought they knew how big one was!

Planning Poker Estimates

Exercise 2 was planning poker, a technique the teams where pretty accustomed to. If you haven’t heard of it before, here is a quick intro. You need a set of planning poker cards for each estimator. Next you need a reference point. Since planning poker is unitless everything is estimated relative to something else. In this case we chose Spain as the reference point and assigned it a value of 3. The item to be estimated is discussed, then each person chooses a card from their set that they think is the closest fit. e.g. if something is nearly twice as big, you would select a 5. Everyone displays their cards at the same time (the idea is that people are not influenced by each other). Then people discuss the outliers, usually the highest and lowest numbers to see if the people who selected them had a different understanding to everyone else. After discussion people re-vote taking into account anything uncovered in the discussion. Once everyone converges on a number that is selected as the estimate. Usually if convergence is not reached by the third vote you take the highest number or majority number.

Here are the results:

Exercise 2: Planning Poker					TIME	4:51
	Group 1	Group 2	Group 3	Group 4	Group 5	Correct
Spain	3	3	3	3	3	3
China	40	40	40	20	40	40
Luxembourg	5	0.5	1	0.5	1	0
Denmark	13	1	2	1	2	1
South Africa	20	5	5	5	5	8
Belize	?	1	?	?	?	1

The discussion this time commented that “it looks like we copied each other“, “somewhat easier to get consensus“, “estimates were closer“. I asked what happened with Belize, and most team admitted they didn’t know where it was or how big it was. I asked them why they didn’t estimate anyway, as they did with the Appalachian Mountain Dog, and we concluded that having a ? in the pack of poker cards made it seem okay to not know. We also talked about whether there was a difference between 100 (large estimate) and ?. If you estimate ? it means you don’t know enough to estimate. If you estimate 100 it means you think it is huge. It’s important for a Product Owner to know the difference. For a ? he can provide more info before he decides it’s too huge to consider. If you use 100 instead of ?, a PO might discard the story as too expensive, rather than having a conversation to see what is not understood.

Affinity Estimation

Exercise 3 was affinity based estimation. This was new to everyone in the session. Here is how it works. You lay out one set of planning poker cards so that you have all the numbers. You agree a reference story, and place it next to the reference point. In this case convertible went next to 3. Then you give the stack of story cards to the first person in the team. Each person has a turn. For your turn you can either: take a new card from the stack of story cards and place it where you think it fits, or you can move a card already on the table to a new spot if you don’t agree with it. You must always move a card if you don’t agree with the estimate rather than playing a new one. So if you play a new card it means you are happy with every card on the table. After your turn of either moving 1 card, or placing a new one, you hand the cards to the next person. If you do a full circuit of the team and one card has been moved each time, then you discuss it before continuing. You can let people explain why they are placing a card at a certain number however, last time I tried to teach this technique people seemed to forget about the don’t debate it unless it moves, so this time I went a bit further and insisted on silence unless a card had been moved by everyone.

Here are the results:

Exercise 3: Affinity Estimation					TIME	4:51
	Group 1	Group 2	Group 3	Group 4	Group 5	Correct
convertible	3	3	3	3	3	3
motorbike	0.5	0.5	1	0.5	1	1
starship enterprize	100	100	100	100	?	100
SUV	8	5	5	8	13	5
minivan	8	8	8	13	5	8
bus	13	20	13	20	20	20

This time someone felt like it took a lot longer. I pointed out that it had actually taken exactly the same amount of time as planning poker. Others commented that “it makes you think more” and that they were “less affected by those around you“.

Relative Estimation

The last exercise is a technique I invented myself, although I think it’s probably a fairly common evolution in estimating. I like to call it relativity  I like it because it focuses people on the relativeness of the estimates over the numbers. Here’s how it works. You take all the story cards and arrange them on the table in ascending size (ignoring the reference point for now). Just decide what is bigger and whats the same size. You can all do this together. Usually there is less debate about what is bigger, the debate is how much it is bigger by.  Once you have arranged all the cards, only then do you assign numbers, starting with the reference point.

Here are the results:

Exercise 4: Relative Estimation					TIME	5:52
	Group 1	Group 2	Group 3	Group 4	Group 5	Correct
impala	3	3	3	3	3	3
elephant	40	40	100	100	100	100
giraffe	13	20	13	10	20	40
elephant shrew	0	0	0.5	0	0	0
crocodile	5	8	20	20	8	13
snake	0.5	0.5	1	0.5	0.5	1

So how did people feel about this one? Interesting one team thought it “sucked” because they “couldn’t agree on the numbers“, however on reflection they agreed that they had implicitly done the relative ranking in the previous exercise. Others felt they had “a clearer idea when ranking first“.

What did we learn

At the end I asked them to look at all the results and see what they noticed. Someone pointed out that the lower the estimate the closer they were to the truth. The implications for our sprints -> Keep the stories small! We also discussed the importance of having the same reference point if we were working on a single backlog with 3 teams. Finally I asked if they would change anything about the way we estimate based on this. The answer: They are keen to give affinity estimation a try in our next estimation meeting.

Try it out

If you want to run this workshop yourself here is what you will need:

• Cards with the items on them for each team
• Planning poker cards for each person
• A flipchart and marker to write up the results of each round for people to see
• A stopwatch or cellphone with timing functionality
• Some team members keen to learn about estimation, I did the workshop with 5 groups of 3-4 people.
• An hour time box is sufficient if your team are used to estimating. Maybe more if your teams are new to it.

I have a PDF you can print out with the cards, and sheets to track the results here: Estimation Techniques

If you try do it, I’d be keen to here about your results.

If you enjoy this workshop – check out more of them in our Growing Agile: A Coach’s Guide Series.