How can a Developer Estimate Times? 227
SubliminalVortex wonders: "Many times in the past, I have been asked on 'how long' it would take to implement a certain features/fixes in a product. What's interesting is that many times, certain 'fixes' is adjusting the wording/placement of the items in question; in other cases, users want the product to do everything they ever imagined, since it already started by following their line of thought. From there, the problem continues. From the user interface, people 'imagine' and think that 'oh, it would be easy if...' and scenarios occur, not only internally from the company using the product, but the clients themselves. Usually, several good ideas are there, but estimating times is a pain in the arse if you have a platform you're writing code for which has no documentation. How do coders estimate times to their bosses? If I know the answer outright, I'll give it, but in some cases, I don't how much time I'll take from other developers *because of the lack of documentation*. I'm going to have to bring in my D&D dice next week just to start."
Design the feature out (Score:5, Insightful)
Use the Estimator's Rule (Score:5, Insightful)
It always takes longer than estimated, even after accounting for the Estimator's Rule.
Experience (Score:5, Insightful)
Well.. (Score:1, Insightful)
YEAH, I posted my first 'good' comment in a while. hehe, karma went from terrible to bad
Anyways, enjoy!
Experience, solid requirements (Score:5, Insightful)
For the most part, your ability to give an estimate on a change is a matter of experience. General experience will give you a rough idea of how long it would take to implement some random feature. Specific experience with the system in question will give you a much more refined ability to make an estimate. If you can't make an estimate, or you can only make a rough estimate, be upfront about it. Tell your manager or the customer (if you're the one working directly with the customer) that your estimate is very rough, and try to add as much buffer time as possible. Also, don't get into granular estimates when you can only give a rough SWAG (Some Wild-Ass Guess). If you think it'll take you an hour, say a day. If you think it'll take you a day, say three. If you think it'll take you three days, say a week (experience will tell you how much buffer to add). If you don't have enough experience with the system to make a good estimate, ask your co-workers for help. It's always better to under-promise and over-deliver than the other way around. Besides, while you may end up over-estimating some feature, chances are you also underestimated another one and the extra buffer time you don't need for feature A will be invaluable to finishing feature B.
To help you make better estimates, you should make sure you get a solid set of requirements from your customers. Time should be spent with the customer to determine exactly what it is they want (they may not be able to articulate it clearly), and whether or not there's already a solution they could use if they're only willing to change their process. Avoid telling the customer at that point whether something is "easy" or "hard" (and definitely ignore them when they suggest something will be easy -- they don't know the inner workings of the code, so how will they know if it's hard or easy? Changing a piece of text may look trivial, but may involve a number of external factors such as localization or legal that would turn a 5 minute fix into a two month battle). Take what you've learned, distill it down into a solid set of requirements, and take that back to the customer for sign-off. Once they've signed off, they don't get to change their mind any more for this cycle (however long that may be -- days, weeks, months, or even years, though hopefully not nearly as long as that). Then you get rough estimates from the developers based on their gut feelings and knowledge of the code base, and have them investigate deeper to solidify those estimates. At that point, you have enough to build a schedule, but allow for buffer time! A developer's 8 hour day may not be 8 full hours of coding. It may be 4 hours of coding, two hours of meetings, an hour for lunch, and an hour of "filler" for the cost of task-switching depending on how the coding hours and meeting hours overlap. Based on your experience with past projects, you should have a good idea of how much time per day a dev actually gets to spend on coding and use that to build your schedule. For example, developer estimates are all in 8 hour day increments, but you know from experience that developers average 6 hours per day of productive work. That means a 5 estimated days will actually take 6.7 real days, so use that as your buffering criteria. (BTW, six hours of productive work is actually high. In most cases, it's really closer to 4 or 5.)
Finally, don't forget QA. Just because a developer has confidence that he can implement a specific feature in one hour doesn't mean it won't take QA a week to verify all of the ramifications of that change. You really need to have your QA team working in lock-step with your dev team. Your dev team should write technical specs on the features, and your QA team should estimate impact off of those. The "agile" disciplines that are en vogue lately try to incorporate testing directly into the development process, which is good. However, that means that developers will be in charge of including QA costs on their estimates, which developers are usually bad about doing
Re:Good 'ol Scotty (Score:3, Insightful)
I think you left something off at the end. "And if you get it done early, you will be rewarded with even less time for the next project."
yer gonna have to spec it out... (Score:3, Insightful)
Re:No Easy Answers (Score:4, Insightful)
I would add, dont let them sucker you into making
an estimate without doing the decomposition, if there
are items that are risky because it is something
newish, allow time ( up front is best ) to do some
empirical development to figure out how it all works,
and if you normally pad estimates with a x 2 factor,
pad this part with a x10 factor.
*Dont* succumb to the temptation to cut back any
estimates once you see what the totals come out
to. Do your estimation at the "leaf nodes" only, and
let the numbers come out as they come out.
And even if the managers dont like the big numbers,
at least *you* will know. They may well tell you you
have less time than you asked for. Dont agree when
they come out with the "it wont *really* take that
long, will it?" stuff. Stand your ground, but realize
you may not get the time. Then you can pre-prioritize
how you execute on the project, so that the most important
stuff is done ( hopefully ) when their stupid deadline
comes up. Now, that does not mean that you should be
disagreeable in your disagreement. But when they
throw on the pressure to bring forward the deadline,
find a way to remind them that you are working toward
that deadline, but you dont think it achievable. Course,
then they will try the "we cant have you lead unless you
buy into our stupidity" kind of stuff. Dont have
an answer to that.
Steps that help (Score:3, Insightful)
o Break the problem down into small pieces. If you have an item on your schedule that says "Do X" and the estimate is "two weeks", you do not understand what X is. Keep splitting it into X.1, then X.1a, then X.1a.ii, until the pieces are all less than three days, preferably much less.
o Clock time != development time. You will spend half your time going to meetings, preparing TPS reports, and making phone calls to find out why your test server is down. To convert development hours to schedule hours, use a factor of two.
o Keep an estimator's handbook. Ever noticed these in a technical bookstore? You can look up how long it takes to plumb a bathroom, build a wood frame wall, etc. Keep your own list of how long it takes to build a credit card form or a bulletin board. Look up the phrase "function point" in ancient literature and see if you get any useful ideas.
Ah, time estimation (Score:5, Insightful)
Time estimation is part of project managment. A good course in project management will give you the tools to effectively deal with these situations.
Short and sweet, when asked to give an estimate (any estimate) ask them if they want the quick and dirty estimate now (which may be off by 1,000% or more) or if they would like a more accurate estimate. If they want the quick and dirty then use whatever method you use now to give them the estimate. If they want an accurate estimate then tell them you'll research the current state of the project, the development effort required for functions x, y, and z, and the resources required to accomplish the effort.
Then follow a good estimating process - there are many available in many books about this subject. In my case generally new projects are very similar to older projects, and are all built on the same foundations so it's easy to start with a list of tasks to get from point A to B, and list the resources and dependencies for each task. Link them all together, and then go to the boss and say, "Assuming that we have these people and these reources, the project will be completed x weeks from kick-off."
When (not if) the boss says, "Well it needs to be done by this date!" Calmy reply with, "We can meet that goal by either scaling the work back, or increasing manpower. Which is more suitable for this situation?" and so on.
The one thing you don't want to do (that all too many software engineers do on a regular basis) is over estimate or underestimate the work required. In either case the customer (generally your boss) learns not to trust your estimates. This is bad on so many levels, and is the main reason that so many software engineers work more than 40 hours per week on a regular basis - especially those that overestimate the time required trying to combat this (paradoxically).
A more accurate estimation process will include a frank discussion of risk and risk management. "This schedule is good as long as we assume the code base is well documented and follows reasonable coding practises." Then find out what the customer wants you to do if you find this assumption to be false. Give them the whole scenario - the risk may be low, but if the assumption proves wrong then perhaps the project schedule or resources just tripled. That may be too great a risk to take even if it's unlikely - so how do we mitigate that risk?
I could go on, but really this is a basic question that millions of people have asked for thousands of years. There are much better and more thorough resources to answer it. Visit your local library if your company isn't willing to send you to a simple three day project management seminar. They won't teach you much that's new, but they'll tie it all together and give you the tools you need to communicate effectively with others - most of the important stuff is being able to say, "I'll get back to you on that by thursday."
-Adam
Re:Design the feature out (Score:3, Insightful)
As a bonus, this way you can write a contract with detailed information on services to be performed and avoid costly feature creep.
Re:Design the feature out (Score:3, Insightful)
Just try to be fair in your estimate; fair for you and fair for the business unit. If your estimate is something that the business side can't understand why it may take so long, tell them in detail what goes into it. The Business side is the one who wants the product and the estimate, so you have to communicate to them clearly why the time is what it is.
DON'T DO IT! (Score:5, Insightful)
Don't do it!
I'm serious. If we've learned anything over the past forty years, it's that trying to estimate software development time is a waste of time. Why? Well, for one things, bugs. To know how long it would take to find and fix them, you'd have to know what the problem was, which means the hard part's done. For another thing, specification flux. Most reasonable project specifications for non-trivial projects don't provide enough information to accurately predict how long it will take to code. And producing such specs, and using them, will take longer than the development would have taken in the first place. And, if that's not enough, think of Dilbert. Specifically, the one where his boss asks him "Will there be any unforeseen problems?"
I once told a client (a dot com) "I'll tell you how long this will take if you can tell me what you stock will be worth six months from now, four months after you IPO. And I'll be more accurate than you will. If you'd rather, you can tell me when the next two-day rainstorm will be." He hemmed and hawed, and basically refused to give any sort of number whatsoever. I said "Good, it looks like we've got a nice honest relationship going here. Neither of us is willing to lie to the other, or claim to know something we don't. So let's start prioritizing these features for the mock up."
Let me say it again: Don't even attempt to estimate development times.
So what should you do instead? Incremental development with frequent opportunities for feedback. Ideally, the users should be able to play with a nightly-build prototype whenever they want from the first week or so on. If they ask for a time estimate, tell them it will be done when they're happy with the result, and don't want anything more, or they're unhappy with the rate of progress, and call the whole thing off.
But don't lie to them, no matter how hard they push.
--MarkusQ
To thine own self be true (Score:1, Insightful)
"To thine own self be true." -- Decide how "tough" the task is in instinctive units. Never reveal these to anyone. This is what you track for yourself. At the end of every project, evaluate how close you were on your "toughness" estimate, and adjust your head accordingly. This has nothing to do with how long it took you to estimate; it has to do with whether the job turned out surprisingly easier or harder. You'll be surprised how quickly you can get good at this estimate.
Keep a time_multiplier. If you have the luxury of working for a single employer in a stable environment, this will quickly converge. Each workplace will need its own calibration however. Also keep this one to yourself. After every project, see what your "hardness * time_multiplier" predicted (hardness after the adjustment above). If you are low, bump your multiplier; if you are high, drop it. This also can converge quite fast.
Now the political parts; much mushier. Never state an estimate before you have a confident feel for the "hardness" number; give "estimate estimates", of the form, "I don't know, but if I drop everything now I can give you an estimate in 6 hours (or 4 days or whatever you need)." Estimate estimates are also easy to get good at, but hard to sell politically. You have to stick to your guns here, or you are sunk.
Once you have your "hardness", and your "time_multiplier", you have what you really think it will takye you to do the job. Now you need to throw in a political_factor. Some bosses will take your estimate and give you 75% of the time (figuring you are padding). Learn to pre-multiply for those.
Some bosses will understand you are projecting the center of a distribution; be very straight with those (and stay with them even against an offer of 120% of your former pay).
Some will "negotiate" with you; I once was successful in one of those negotiations when I said, "I thought you wanted my best estimate. If you don't, just make up any number you like; it won't affect how quickly the job gets done." Be very careful to distinguish these people from those who are willing to change the task to fit a schedule. For those people, you may have to hit them with another "estimate estimate," but switching details usually quickly gets to wher you can do some on your feet.
Summing up. Track "hardness" and avoid connecting it to any "real" idea of time. Keep your instinct of "this should take a day" and learn separately how long it takes you to do what you think of as a day's worth of work. Track "workplace factors" collectively separately; this covers things like how many meetings happen, how often you rebuild your machine, .... Finally, track what you need to do to your true estimate to "sell" it.
Selling may be inflating in expectation of an automatic deflate. It may have to do with how your bonuses are calculated. It must include lots of factors about what you tell your boss. Finally, it can also include a cost/benefit analysis ("If I say 78 hours and I come in at 83, we lose our shirts, but if I say 78 and I come in at 72, we don't get the contract"). Sometimes low is too expensive so you pad, sometimes high is too expensive so you shrink.
Dependent variables... (Score:3, Insightful)
http://www.goldratt.com/ [goldratt.com]
There are two keys to delivering almost always on time; Planning and Management.
Planning means thinking the project through in the best detail before you start coding. Agile programming seems to work in short, quick-and-dirty projects, but my experience is that it falls short if the project is large and complex. The less decisons you have to make in the middle of the project, and the less changes you have to make as a result, means that the work can me seen to progress along the planned path.
Management means controlling the variables, and that's where the techniques in "Critical Chain" shine. Nothing, absolutely nothing, is allowed to delay the critical chain in the plan. The biggest dividends come from two parctices: Continuity and Resource management.
It used to be that if I had a week to do a job that I knew I could do in a day or two, I'd do whatever else needed doing and start two days before the task was due. Of course, If something came up, (essential co-worker gets sick, computer crash, virus, software needs updates, whatever) then I might be delayed during that two days and the task is delivered late. If the task was on the critical path, then the whole project becomes late. The first 10% delay on a project typically costs 25% of the profitability, the next 10% late costs 20%, so being only 20% late can cost 45% of the profitability. People tend to think that if they have some slack, they can relax. Then they wonder why they are late. They should work as if there was no slack! Relax at the END, not during.
In a large project with multiple developers, some of them may be required to cordinate along different paths, not just the critical path. This is why Godratta called it the "Critical CHain". Those resources MUST be focussed on keeping the essential workflow from bogging down anywhere that may delay the longest chain of dependencies.
This idea came out years ago, and I've been following project management ever since the 60's. This is the first big change I've seen since 1969. Everyone who told me they tried the Critical Chain method and it didn't work, didn't follow the process. Siemens practically took the whole market for 7200 rpm hard drives by developing them first using the Critical Chain method.
Good luck on your projects.
Re:Design the feature out (Score:4, Insightful)
As for comparing time- first, you have prior experience. Having done something similar on another project can give you a good idea as to time. Then there's similar things to another part of the program. Finally, just because something is similar doesn't mean the code can be shared. But it does give you a yardstick for time.
There never should be a "similar task" (Score:3, Insightful)
But... ever since the days in the 1950s when the subroutine was first invented, there never should be such a thing as a "similar task." Anything that was done before should have been packaged as a subroutine. Or a reusable object. Or a programming system (entire library or language) designed to attack that class of task.
Managers that prize predictability over productivity assign their people to do trivial variations of the same work over and over again and get a predictable straight-line curve of time versus work accomplished.
Good managers allow their people to invest time into tools and techniques are new, hence risky, but pay off with a multiplier effect when they work. They get an unpredictable but exponential progress curve.
(Bad managers, of course, won't accept the notion that a new task should be estimated at the time that a similar task did in the past. They insist that the task should be estimated as if everything were going to go perfectly this time, no missteps, no problems. ("It will take about a year." "Why do you say that?" "Because, remember, the last time we developed the frammis for the foithboinder project, it took a year" "Yes, but a lot of things went wrong. We had several false starts, because the original spec we were given was wrong. And Jim left halfway through the project and it took a while for Kathy up to speed. And we lost a month when we changed over to the new source code control system." "And why do you think nothing will go wrong this time?")
Re:DON'T DO IT! (Score:5, Insightful)
You are completely missing the point here. It is precisely because it is important that you should not fall into the trap of lying to people. Instead, you should do what any other responsible R&D team would do--decouple the processes of R, D, and whatever comes after.
Here's how it works:
The point is, because it's important, don't let it become an all or nothing proposition. Make sure that, on whatever date they need to, they have something to run with, and it is as close as you can come to the best possible version that could be produced by that date.
And it will likely be better than it otherwise would have been, because you can apply all the time you would have wasted defining and defending schedules to actually keep the project moving forwards.
--MarkusQ
Re:DON'T DO IT! (Score:1, Insightful)
Get real.
Software development increasingly a commodity. Competition is increasing; prices are normalizing; development tools are improving, making basic programming more accessible / less mysterious; and, more and more, the work can be done from anywhere. Big companies have figured it out (out-sourcing), and I even see trends that local, smaller businesses are figuring it out as well.
Yes, there are people out there innovating on the edge. There are people creating new algorithms, developing new protocols, developing new technologies, even reverse-engineering systems and software, and so forth. Probabilistically speaking, you're not likely one of them; rather, (1) you've been tasked with creating something new similar to things that have been done hundreds of times before ("Build me a website"), (2) you've been assigned a module or a subroutine with specified interface and performance requirements, or (3) you have been selected for the more dubious honor of maintaining or changing someone else's work.
If you're in case (1) and you can't estimate, then you're probably either inexperienced or you're using this opportunity to learn and try out the latest and greatest new technology on your client's dime.
If you're in case (2), you're probably on a big project and as a programmer, estimating isn't your responsibility. If it was in your responsibilities, you may have had "PMP" stamped on your head and may have been trained in your company's CMM-approved process---then you wouldn't be asking. Rather, you may be bitching about your program manager or project lead asking you to estimate. Be grateful that you've been asked for your input at all.
Case (3) is probably the worst. Neither an individual programmer nor a company of engineers has strong incentive to make it easy for some other programmer or company to get paid for follow-up work. But guess what? You have an upper bound on any estimate: It's the time it would take you to rewrite, and you should be able to estimate that.
So, play it up while you can. Are you really so special that I shouldn't hire or contract with the person or company that *can* estimate and can provide a track record of on-time / on-budget work instead of you?
-JM
Re:Design the feature out (Score:5, Insightful)
Newbie : 8x as long as estimated.
Seasoned Developer : 3x as long.
Elite Veteran : 2x as long.
Here's the rule I use when dealing with developer estimates (copied from a post almost a year ago) :
Time estimage guidelines:
New programmer fresh out of college: Take his estimate and multiply by 8x. Yes he could get it done in 1 day, assuming he got so cranked up on caffeine his eyes stopped blinking and he worked on that (and nothing else) for 24 hours straight. In the real world a newbie can dedicate about 2 real hours doing a particular task each day, the rest is spent coming up to speed on corporate coding standards and libraries, email, breaks, and not 'in the groove'. Also, you are expecting him to document it but he didn't account for that in his estimate - his estimate was only for time to code the actual lines of code.
Veteran programmer of average skill, single person project : multiply his estimate by 3x. A third of his day is spent hand-holding the newbie, and another third is spent hand-holding management. The other third is spent programming, but luckily he knows to pad the schedule some (not enough, but some.)
Veteran programmer of uber skill, single person project : multiply his estimate by 2x. This is as good as it gets. A uber veteran programmer knows to leave his email client closed and his door closed so he can stay in the zone. He knows to pad the schedule more than he really thinks he should. And it still takes him twice as long as he expected.
Multiple people working on the same project : increase the timeline by a factor of 1.2 per additional person. If two people ought to be able to do it in 10 days it will take 12. If 11 people (10 additional) ought to be able to do it in 10 days it will take
Re:Design the feature out (Score:3, Insightful)
Ah, yes. Stealth requirements. The seasoned developer will be familiar with the practice of requirements which you didn't mention, and when asked about them, will say, "Well, that goes without saying!". And it's true, it *did* go without your saying it...
Re:Management Solution (Score:2, Insightful)
Re:Scrum Development Process (Score:3, Insightful)
Do not confuse complexity with unknowability. A sulfuric acid manufacturing plant is complex, but every part of the process is known, and is known to be repeatable; and thus it is reasonable to ask how long it will take to build a proposed sulfuric acid manufacturing plant and how much it will cost.
Scrum is a development method for nontrivial software projects with many unknowns and little repeatability. Such projects cannot be estimated as if they were complex-but-known, no matter how strongly the project sponsors wish they could. The best that can be achieved is to control risk with short timeframe subprojects (no more than thirty days) and daily formal sessions (no more than 15 minutes) where the developers (seven plus or minus two) address only what they did yesterday, what they plan to do today, and any impediments. The best the requesters can do is to know that the developers are following a procedure that will surface problems early, tend to build the most useful parts of the system early, and provide frequent opportunities for continue / discontinue decisions during the process.
Estimates are only as good as what you know (Score:2, Insightful)
Primarily, it depends on what you know. There are a number of techniques beyond the W.A.G. of developer's experience with the software at hand (a google search for "estimation techniques" turns up over 900,000 matches), and depending upon the circumstances one technique may be better suited than another. IMHO they all boil down to one simple fact: the more you know the more accurate an estimate can be. As a manager, you develop a feel for which of your developers pad their estimates and by how much. When the numbers seem high, it is usually because they don't have enough information to make a more reasonable "guess". As a developer you don't want to get caught short when your "rough estimate" turns into a schedule deadline, so you take into account all the things that are likely to come up.
For me, the best way to stay out of trouble is to offer a range in your estimates. The less you know the wider the range and vice versa. You can always qualify the range estimate based upon what you know. Other posts have suggested taking time to "get back to you with that estimate" and that is a very valid technique to "know more", thus reducing the range of your estimate. If you have reasonable management and business users, they will understand that an estimate is a "best guess given the information available" rather than a due date to be slapped on a schedule.
How to come up with accurate estimates, isn't an easy question to answer. If it were, there would be one or two simple ways to do it accurately, and everyone would be following those procedures to come up with estimates.
FWIW...
Old Boss's Forumla (Score:3, Insightful)
He asked each of the software engineers how long it would take, based on their "gut" instincts.
Then he multiplied the figure by two and moved it to the next higher order of time. I.E. weeks become months, months become years. One time he asked us to estimate when we could get something accomplished and we told him two years. He cried for a little bit and went to upper management and said "Never, we can't get that done".
Surprisingly this turned out to be a very effective way for him to gague when things would actually get accomplished, and more often than not he was dead on correct. He also didn't let me in on his secret until just before he quit, but it has also helped me to make my own estimates for customers when I'm doing contract work now. I do the same thing where I make a preliminary estimate based on known issues and then apply the above rule to my own guess. I have (fortunately) been fairly accurate with my guesses at that point, even if it makes the customer angry with them insisting that it must be done in a shorter period of time.
Not an honest answer yet (Score:2, Insightful)
30 seconds means 15 minutes.
10 minutes means 5 hours.
12 hours means 6 days.
14 days means 7 weeks.
8 weeks means 4 months.
6 months means 3 years.
* First Rule of Project Management: Divide estimate by two, and round up to next higher unit.