(Taking you seriously) Try keeping a log of your estimations and the actual time taken. Write down both the original two day estimate and the extended 11 day one.
Then after a few months you should have a good set to look at and hopefully convince yourself to believe your modified estimates. Or how to adjust your formula to be more accurate.
I know a management consultant that has a surefire method of stretching a sentence into a book. He is also good at stretching the truth. I believe he helped Tim Ferris out with the four hour workweek
I once knew a project manager who took 2 pages I'd scribbled while explaining him the possible architecture of some odd project, and came up with a 150 page SRS
I think you want the unit I always round up to: sitting-in-your-source-code-folder-unviewed-for-a-year-with-all-intentions-of-finishing-it-though-really-just-avoiding-it-because-you-know-it-is-way-too-large-to-really-finish-to-the-degree-of-polish-you-initially-envisioned.
Really the point is to keep yourself honest and realize that things tend to take a lot longer than you'd expect.
Also, if you are scoping a project out in terms of years you probably want to break it down into smaller features and implement them one-by-one.
Of course it has a few bugs: It isn't really polished, is easy to game, vulnerable to attacks, a security hazard, doesn't work in IE6, isn't scalable, dosen't verify links properly, sometimes misses dupes, doesn't have a proper admin interface and needs some design love.
But except for that, and a few other things I can do in a weekend it's done.
That's exactly the point though. You're never done with a software project, so define "done" in a way that gives you something maximally useful in a minimum amount of time.
The "I can do it in a weekend" meme is just ruthless application of the 80/20 principle. Pick out the most useful 80% that'll take the shortest amount of time to implement. Then repeat until you've got the time down to a weekend. Chances are that if you've picked well, you've still got something fairly useful. Most decent software ideas have a useful kernel that really can be done in a weekend.
Gall's Law: “A complex system that works is invariably found to have evolved from a simple system that worked. The inverse proposition also appears to be true: A complex system designed from scratch never works and cannot be made to work. You have to start over, beginning with a working simple system.”
Once you've got folks using your system, then you can worry about all the nitty-gritty details. But generally, it doesn't matter.
Sometimes even when I define what done means before I start a project, I embarrass myself with my underestimates of complexity. Software is just plain fickle. Proof of concept on most any web app? Weekend. Something that satisfies anyone but me? At least two weekends :)
The best time to define done is when the time period has expired. Just take what you've done, list that as your acceptance criteria, and viola! You're batting 100%.
The world needs more people saying foolhardy things like "I could do that in a weekend", not less.
Actually it occurs to me it would be incredibly interesting to see someone's attempt at cloning justin.tv in a weekend. Sure it's impossible, but I would be fascinated to see what got done if someone actually tried.
He has a point, but anyone who has ever built something non-trivial from start to finish already knows this. It's the subtleties, not the broad concept, that drains time.
But it's the concrete knowledge that your prototype just needs polish that makes the "weekend hack" valuable. Given the choice between two developers with proposals, one of whom spent last weekend banging out a prototype but with no plan, and the other with an elaborate plan and documentation but no code, which would you pick?
That's why the "weekend" meme persists. It's not about being "done" (software is never "done" anyway), it's about getting it to a point where "done" has meaning.
The little stuff can always be done. But my fear is that the people who insist on mocking the weekend hackers are the people who refuse to start on the big thing for fear that they'll take too long.
Spikes (in the agile nomenclature) are great fun. You just try to figure out what the shortest point from where you are to minimal functionality is, then get there.
Regarding a HN clone:
Sorting? Just do it by date.
Logins? Just a basic login, perhaps with tons of security vulnerabilities.
Flagging? Later
Comments? Not yet.
Once you have that, which is usually just a weekend job, then you can start to see what's going to be necessary and what isn't and start to see some of the edge cases and features you didn't even know you would need.
While his specific point is true, it is also true that delusion can be a valuable tool of progress (aka if we knew what having children is really like, the humankind may have died out). Obviously, you can't hash out a full-fledged system in a weekend, but you can get enough of a start that it pulls you in and makes you work on it after the weekend is done. Also, sometimes you can actually build a pretty sophisticated project if you have done enough of the pieces in the past. Of course, all of the above applies only to self-inflicted work; "managerial pressure" should be resisted :)
I think a useful place to begin estimates, before you dive into the detail level and go bottom up from tasks, is estimate by analogy. Look at how long it has actually taken you or your team to deliver a real system. Use that as the baseline for the estimation process.
Unless you have delivered systems in 2 days, you probably don't have a process that supports delivering systems in 2 days. That's why delivering something in 2 days is a good thing to try. I like this post of a team building something in 4 hours.
http://blog.palantirtech.com/2009/07/06/the-multisnake-chall...
I understand the annoyance at someone saying they could replicate a product in a weekend. And I also understand the desire to say that this is a bad idea because it probably isn't possible.
However, I think all of us at some time or another worked up the courage to build something much larger, important, or interesting because we deluded ourselves into the belief that it would be simple or easy or only take a small amount of time.
I consider this nearly universal estimation problem a benefit to the entrepreneur (at least at the beginning). It allows people to start projects that would be too daunting if they saw each and every task in front of them and that it would actually take them many months or years of their lives to make it work.
