Software Estimation: Remember that Targets are not Estimates
I was asked recently to help in creating some estimates. Dates were mentioned, features were requested, but generally the information given was pretty vague, as is common with many estimates. So, we start digging. What is needed, when is it needed, in what form is it needed? What does success look like?
There's lots of estimation tools out there. Steve McConnell's Construx Estimate (even though it's written in VB6 and was created in 2001) is well thought of, and can certainly get one jump-started with an estimate. Of course, garbage-in, garbage-out still applies.
Patrick's been using the PlanningPoker technique for estimating lately. It's a useful technique that relies on the folks doing the work also doing the estimating. (Always nice.)
If you take a look at the White Papers section of the Construx website (free registration required, but it's worth it) you'll find a number of excellent presentations in PDF format that are good reminders and primers when dealing with daunting Software Estimation tasks.
Pick up Steve McConnell's book "Software Estimation: Demystifying the Black Art" if you want a nice introduction to the voodoo that is estimating.
I've been reminded of a few important tenets doing this estimating processes.
Know your scope
Negotiating the scope of your project, the features, subsystems, etc. with your business stakeholders is a good first step in getting your head around "what are we really trying to accomplish." User stories and complete and accurate Use Cases are invaluable when trying to nail down how large something is.
Targets are not Estimates
Folks sometimes come up with dates, like "February 2008" as a target for a project to hit. After a while that target date starts getting treated like an estimated date. It's important that everyone on a project remember (or be reminded regularly) that targets are not estimates. If you want to hit a date, you'll likely not know when if an estimate is good enough to hit a target until you start moving far enough into the Cone of Uncertainty.
Ask for Worst Case - then Double That
Jeff Atwood points out, after reading Steve's book, that asking for multiple points (best case, worst case) when asking folks for an estimate is important. He quotes Steve, emphasis mine:
Considering that optimism is a near-universal fact of human nature, software estimates are often undermined by what I think of as a Collusion of Optimists. Developers present estimates that are optimistic. Executives like the optimistic estimates because they imply that desirable business targets are achievable. Managers like the estimates because they imply that they can support upper management's objectives. And so the software project is off and running with no one ever taking a critical look at whether the estimates were well founded in the first place.
This quote nails it for me. No one wants to give a realistic estimate. It's hard and sometimes it's potentially career-limiting. Steve quotes Fred Brooks (who, as a random aside, is the uncle of a good friend of mine from high-school):
It is very difficult to make a vigorous, plausible, and job-risking defense of an estimate that is derived by no quantitative method, supported by little data, and certified chiefly by the hunches of the managers.
— Fred Brooks (1975)
A few companies encourage worst-case estimates in the hopes that you can look like a hero if you make it happen. A few companies ago I worked somewhere with the philosophy: Underpromise, over-deliver. It was the founder's belief that this idea was fundamental to making happy customers. It seemed like a good idea, except when you actually delivered and things turned out to be simply: Promised, delivered. Which isn't all that bad, actually, considering that folks say that fewer than 1/4 of projects actually do deliver on time.
Learn From the Past, and Don't Forget It
There's a great slide in the 10 Keys to Success PDF up at Construx that compares 120 projects at Boeing, juxtaposing those that were estimated with Historical Data and those that were done without. Seems obvious, but it's useful to see these kinds of things and "learn from the sins of our fathers."
When estimating based on historical data, however, sometimes people use the historical estimate rather than the historical actual. In fact, a lot of companies don't closely track the actuals. You'll be better off if you not only keep the actual datas, but also compare the original estimate with the actual result in a project post-mortem.
- How do you estimate your projects?
- Do you estimate?
- If so, what tools do you use?
- Do you use Function Point Counting?
- Do you use other techniques?
- What does success look like for your project?
- What's your success rate?
Discuss, dear reader.
About Scott
Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.
About Newsletter
I used the proxy-method of estimating based on the project actuals of the only similar project we had done (which was smaller in scope and complexity). I applied "T-Shirt" sizes (Small, Medium, Large) to map development effort for different parts of the deliverable.
Where the provious project had , say five "Medium" data tables, the new project had 20 "Medium" data tables, and so on...
This gave me an effort estimate figure of several hundred days.
I presented my estimate internally and I was called "unprofessional and irresponsible"! When I explained how T-Shirt sizing works, my manager created a new T-Shirt size ("Tiny") and wanted most of the project re-estimated with it.
My manager then explained that he was looking for a figure of 40 to 60 days! "In my experience", he added, "No piece of software takes more than about one hundred days!".
Note: I'm looking forward to starting development work with my new employer in a few weeks. Reading Steve McConnell's book "Software Estimation: Demystifying the Black Art" and using the techniques it explains has really helped my career!
I do sometimes use (depending on the vagueness of requirements) the 'take the worst case and double that' technique.
Planix looks very interesting. It's very similar in usability to how I use Excel/Project to do estimates (depending on the size/complexity of requirements), only easier.
Success Rate? Well so far, I haven't had a project which has gone in loss or not delivered on time. I have estimated projects from 1 man month of effort to over 100 man months of effort. Plain lucky, but i think experience helps...
Cheers.
Lessons learned:
- Never mention hours in a casual conversation
- When asked how long, or how easy? I respond with questions, lots and lots of questions.
- If I give a wrong estimate, it will cost me a raise.
- When I get my final numbers. Add at least 10% more time.
Now I don't give any estimates without writing out all that will be delivered with the estimate. So that when feature creep happens I am able to get an updated budget and extended deadline.
Also, I have setup a system to track estimates against actual time to be able to better predict the levl of effort required.
You had me up until "(even though it's written in VB6 and was created in 2001)". At that point, all I could think was "So what?" Why does it matter if it's written in VB6???
2. Programmer (afraid to be honest) gives an optimistic target instead.
3. Manager gets mad, and asks for a shorter deadline after making threats.
4. Fearful programmer agrees to impossible, hoping to keep job.
5. Manager reluctantly accepts.
6. Programmer delivers a compromised, rushed product late (but ahead of optimistic target)
7. Several months of testing and support past the honest pessimistic deadline the product actually starts working.
8. Manager gets mad again, complains programmers cannot be trusted, and you should always take the worst case estimate and triple it. Gives himself a 20% pay raise and generous bonus for the overwhelming success of the company being first to market.
9. Everyone looks at the success of the company and believes the manager when he says software is always late.
They have the potential to cumulate Actual-Time-Taken-For-Each-Use-Case (or User story, depending on the methodology used) over a period of time for all past projects of the organization and can act as solid collaborative knowledge base of estimating similar use-cases (and in turn similar projects) over a period of time.
A bug tracking system with time-taken-to-fix-per-module type features can help analyze how many bugs a use case had and how much time it took to fix them so that end-to-end estimations for similar use-cases become possible.
When the requirements are very different from past use-cases done in all projects, I just pick the closest one that comes to mind. The plus or minus in that scenario is done mostly on gut-feeling.
I’ve also got into a practise of having two people - me included (at times it's two teams depending on how complex the estimation is) - estimating the same project independently. If we end up with a 5% difference that’s fine; we talk and finalize. If it’s a bigger difference we get-together, talk, go back and re-estimate till we get it right (i.e. till both the estimates are in the 5% range).
What I like about it is that it uses "use cases" as input to the estimate model. After using it for several years and feeding back into it our actual versus estimated, it is suprising how well it works.
You can download it from this linked article, "Size Does Matter": http://softwareindustrialization.com/DisplayPost.aspx?PostID=25
Cheers,
Sunjay
Comments are closed.
I started using Wideband Delphi in small project teams (3-6 members) after reading the article below. I wouldn't say my estimates were perfect but it did help in reducing the (+/-) factor from the estimates.
http://msdn.microsoft.com/msdnmag/issues/07/01/EndBracket/default.aspx
It does appear to be very simple to be true and accurate but it works in small teams. The might run into problems as the team size increases, I guess.
Cheers
Hemil.