Scott Hanselman

Software Estimation: Remember that Targets are not Estimates

May 09, 2007 Comment on this post [12] Posted in Programming
Sponsored By

I was asked recently to help in creating some estimates. Dates were mentioned, features were requested, but generally the information given was pretty vague, as is common with many estimates. So, we start digging. What is needed, when is it needed, in what form is it needed? What does success look like?

There's lots of estimation tools out there. Steve McConnell's Construx Estimate (even though it's written in VB6 and was created in 2001) is well thought of, and can certainly get one jump-started with an estimate. Of course, garbage-in, garbage-out still applies. 

Patrick's been using the PlanningPoker technique for estimating lately. It's a useful technique that relies on the folks doing the work also doing the estimating. (Always nice.)

If you take a look at the White Papers section of the Construx website (free registration required, but it's worth it) you'll find a number of excellent presentations in PDF format that are good reminders and primers when dealing with daunting Software Estimation tasks.

Pick up Steve McConnell's book "Software Estimation: Demystifying the Black Art" if you want a nice introduction to the voodoo that is estimating.

I've been reminded of a few important tenets doing this estimating processes.

Know your scope

Negotiating the scope of your project, the features, subsystems, etc. with your business stakeholders is a good first step in getting your head around "what are we really trying to accomplish." User stories and complete and accurate Use Cases are invaluable when trying to nail down how large something is.

Targets are not Estimates

Folks sometimes come up with dates, like "February 2008" as a target for a project to hit. After a while that target date starts getting treated like an estimated date. It's important that everyone on a project remember (or be reminded regularly) that targets are not estimates. If you want to hit a date, you'll likely not know when if an estimate is good enough to hit a target until you start moving far enough into the Cone of Uncertainty.

Ask for Worst Case - then Double That

Jeff Atwood points out, after reading Steve's book, that asking for multiple points (best case, worst case) when asking folks for an estimate is important. He quotes Steve, emphasis mine:

Considering that optimism is a near-universal fact of human nature, software estimates are often undermined by what I think of as a Collusion of Optimists. Developers present estimates that are optimistic. Executives like the optimistic estimates because they imply that desirable business targets are achievable. Managers like the estimates because they imply that they can support upper management's objectives. And so the software project is off and running with no one ever taking a critical look at whether the estimates were well founded in the first place.

This quote nails it for me. No one wants to give a realistic estimate. It's hard and sometimes it's potentially career-limiting.  Steve quotes Fred Brooks (who, as a random aside, is the uncle of a good friend of mine from high-school):

It is very difficult to make a vigorous, plausible, and job-risking defense of an estimate that is derived by no quantitative method, supported by little data, and certified chiefly by the hunches of the managers.
Fred Brooks (1975)

A few companies encourage worst-case estimates in the hopes that you can look like a hero if you make it happen. A few companies ago I worked somewhere with the philosophy: Underpromise, over-deliver. It was the founder's belief that this idea was fundamental to making happy customers. It seemed like a good idea, except when you actually delivered and things turned out to be simply: Promised, delivered. Which isn't all that bad, actually, considering that folks say that fewer than 1/4 of projects actually do deliver on time.

Learn From the Past, and Don't Forget It

There's a great slide in the 10 Keys to Success PDF up at Construx that compares 120 projects at Boeing, juxtaposing those that were estimated with Historical Data and those that were done without. Seems obvious, but it's useful to see these kinds of things and "learn from the sins of our fathers."

When estimating based on historical data, however, sometimes people use the historical estimate rather than the historical actual. In fact, a lot of companies don't closely track the actuals. You'll be better off if you not only keep the actual datas, but also compare the original estimate with the actual result in a project post-mortem.

  • How do you estimate your projects?
  • Do you estimate?
    • If so, what tools do you use?
  • Do you use Function Point Counting?
    • Do you use other techniques?
  • What does success look like for your project?
    • What's your success rate?

Discuss, dear reader.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Hosting By
Hosted in an Azure App Service
May 09, 2007 5:09
Hi Scott,

I started using Wideband Delphi in small project teams (3-6 members) after reading the article below. I wouldn't say my estimates were perfect but it did help in reducing the (+/-) factor from the estimates.

http://msdn.microsoft.com/msdnmag/issues/07/01/EndBracket/default.aspx

It does appear to be very simple to be true and accurate but it works in small teams. The might run into problems as the team size increases, I guess.

Cheers
Hemil.

May 09, 2007 9:23
I was asked to do an estimate - convert an existing application into a WPF-based one.

I used the proxy-method of estimating based on the project actuals of the only similar project we had done (which was smaller in scope and complexity). I applied "T-Shirt" sizes (Small, Medium, Large) to map development effort for different parts of the deliverable.

Where the provious project had , say five "Medium" data tables, the new project had 20 "Medium" data tables, and so on...

This gave me an effort estimate figure of several hundred days.

I presented my estimate internally and I was called "unprofessional and irresponsible"! When I explained how T-Shirt sizing works, my manager created a new T-Shirt size ("Tiny") and wanted most of the project re-estimated with it.

My manager then explained that he was looking for a figure of 40 to 60 days! "In my experience", he added, "No piece of software takes more than about one hundred days!".

Note: I'm looking forward to starting development work with my new employer in a few weeks. Reading Steve McConnell's book "Software Estimation: Demystifying the Black Art" and using the techniques it explains has really helped my career!
May 09, 2007 10:14
I regularly do estimations at my job for new projects (or new modules in existing projects). This does involve asking back a lot of questions to the clients, but since most of these estimations are done at a stage where we are looking to bag a project, there is rarely ever time for Function Point Calculations. I depend on a lot of experience on doing past estimations and seeing how accurate they were. Also helpful is the knowledge of real projects, their requirements, and how long they took to complete.

I do sometimes use (depending on the vagueness of requirements) the 'take the worst case and double that' technique.

Planix looks very interesting. It's very similar in usability to how I use Excel/Project to do estimates (depending on the size/complexity of requirements), only easier.

Success Rate? Well so far, I haven't had a project which has gone in loss or not delivered on time. I have estimated projects from 1 man month of effort to over 100 man months of effort. Plain lucky, but i think experience helps...

Cheers.
May 09, 2007 10:24
I have some experience with an estimating tool called SLIM. It basically requires before using to have some historic actuals from past projects. The tool, like others, asks input for total resources available, the timeline, and amount of work to be done. The great part about the tool is that as you use it, you continually calibrate it. The first project I used came up with a surprisingly accurate assessment of the work we would do based on the information I gave it, coupled with the algorithms present in the application that account for resource efficiency, bug producing rates, and my favorite: how the laws of diminishing returns are actually accounted for in their forecast for your project. It is great to take results like this to upper management and show them that throwing bodies at a project will actually make the project take longer and cost more. It is very useful, but requires dedication to measuring your actual "actuals".
May 09, 2007 19:23
I recently worked and am still working on a project that has been a thorn in my side. In a random conversation I was asked how long it would take to convert an existing unfinished barely functional Access UI application into a .Net winforms application. I responded in casual conversation that it would take about 60 hours. 300+ hours later and 4 months late the project is still going through changes. The application is 10 times larger and 100 times more functional then his previous version and the customer is still happy because we were only able to bill him the initial 60 hours. At least now, with any changes I can bill him for any changes.

Lessons learned:
- Never mention hours in a casual conversation
- When asked how long, or how easy? I respond with questions, lots and lots of questions.
- If I give a wrong estimate, it will cost me a raise.
- When I get my final numbers. Add at least 10% more time.

Now I don't give any estimates without writing out all that will be delivered with the estimate. So that when feature creep happens I am able to get an updated budget and extended deadline.

Also, I have setup a system to track estimates against actual time to be able to better predict the levl of effort required.
May 09, 2007 21:13
I use a variation of Joel Spolsky's Painless Software Schedules (http://www.joelonsoftware.com/articles/fog0000000245.html), currently in Excel. My estimates have been relatively successful, usually low by 10%.
May 09, 2007 21:24
Scott,

You had me up until "(even though it's written in VB6 and was created in 2001)". At that point, all I could think was "So what?" Why does it matter if it's written in VB6???
May 09, 2007 22:03
Coleman - Sounds like I didn't have you very long! :) I was a VB3/5/6 programmer for many years and I've had a number of downloaded VB6 applications (just me personally) mess up my system by overwriting various system DLLs without asking. It's more of an installer nit, than a VB6 nit, but there it is. Are you a VB6 fan?
May 09, 2007 23:30
1. Manager asks programmer for a worse case scenario.
2. Programmer (afraid to be honest) gives an optimistic target instead.
3. Manager gets mad, and asks for a shorter deadline after making threats.
4. Fearful programmer agrees to impossible, hoping to keep job.
5. Manager reluctantly accepts.
6. Programmer delivers a compromised, rushed product late (but ahead of optimistic target)
7. Several months of testing and support past the honest pessimistic deadline the product actually starts working.
8. Manager gets mad again, complains programmers cannot be trusted, and you should always take the worst case estimate and triple it. Gives himself a 20% pay raise and generous bonus for the overwhelming success of the company being first to market.
9. Everyone looks at the success of the company and believes the manager when he says software is always late.
May 09, 2007 23:36
I’m involved with estimations at work once-in-a-while. I’ve always felt that Timesheets entry systems and bug tracking systems (if used accurately by developers) can end up becoming Estimation devices if they have timelines on a per-task-basis built into them.

They have the potential to cumulate Actual-Time-Taken-For-Each-Use-Case (or User story, depending on the methodology used) over a period of time for all past projects of the organization and can act as solid collaborative knowledge base of estimating similar use-cases (and in turn similar projects) over a period of time.

A bug tracking system with time-taken-to-fix-per-module type features can help analyze how many bugs a use case had and how much time it took to fix them so that end-to-end estimations for similar use-cases become possible.

When the requirements are very different from past use-cases done in all projects, I just pick the closest one that comes to mind. The plus or minus in that scenario is done mostly on gut-feeling.

I’ve also got into a practise of having two people - me included (at times it's two teams depending on how complex the estimation is) - estimating the same project independently. If we end up with a 5% difference that’s fine; we talk and finalize. If it’s a bigger difference we get-together, talk, go back and re-estimate till we get it right (i.e. till both the estimates are in the 5% range).
May 11, 2007 22:56
I have used an estimation model from Geri Schneiders excellent book called, "Applying Use Cases: A Practical Guide." http://www.amazon.com/Applying-Use-Cases-Practical-Guide/dp/0201309815

What I like about it is that it uses "use cases" as input to the estimate model. After using it for several years and feeding back into it our actual versus estimated, it is suprising how well it works.

You can download it from this linked article, "Size Does Matter": http://softwareindustrialization.com/DisplayPost.aspx?PostID=25
May 13, 2007 22:22
Scott, thanks for the review. The stories/pain on this blog is *precisely* what led us to deploy Planix as a tool others could use outside of our company. In the spirit of Scott's blog, we will provide a "magic" account to the first 10 people who contact me at spandey@planixonline.com. Happy planning and estimatiing.

Cheers,
Sunjay

Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.