Why big software projects overrun, and how to avoid it

Most discussions of project risk quote an average overrun. Averages hide the danger. In a study of 1,471 IT projects, Bent Flyvbjerg and Alexander Budzier found that while the average cost overrun was around 27%, that number concealed a fat tail: one in six projects was a 'black swan' with a cost overrun of 200% and a schedule overrun of almost 70%.

That shape, most projects roughly on track with a minority failing spectacularly, is what makes large, all-at-once builds so dangerous. The expected value looks tolerable right up until you are the one in six.

The Standish Group's long-running CHAOS research points the same way. Across decades of data, only a minority of software projects succeed on the original scope, time, and budget, while the majority are challenged or cancelled. Critically, success rates fall sharply as project size grows.

The lesson is not that software is hopeless. It is that size itself is a risk multiplier, and that betting a modernisation on a single large release stacks the odds against you.

Delivering in small, independently shippable increments is the most reliable way to flatten that fat tail. Each stage produces something usable, so value lands continuously rather than at a distant finish line. Just as important, it caps the blast radius of any single wrong assumption.

A staged programme can be stopped, re-scoped, or redirected after any increment. A big-bang rewrite offers no such exit, which is why so many of the black swans are rewrites. Modernising one seam at a time is not just cleaner engineering; it is a deliberate strategy for staying out of the tail.