If you are weighing up what to do about a core system, someone has probably told you that 70% of modernization projects fail. It is a useful number for a vendor and a frightening one for a board. It is also, as far as anyone can trace it, not the result of any study of modernization projects.
The real evidence is less tidy and more useful. Large software projects do miss their targets more often than they hit them, but the risk is not evenly spread. It concentrates in projects that are big, long and built on assumptions about what the old system does. The projects that land tend to be small in scope, start from evidence rather than a wish list, and measure a pilot before they commit.
First, the 70% figure
The trail runs back to a 2000 Harvard Business Review article by Michael Beer and Nitin Nohria, Cracking the Code of Change, which opens with "the brutal fact is that about 70% of all change initiatives fail." No study is cited. McKinsey repeated it in 2015 in Changing change management ("70 percent of change programs fail to achieve their goals"), again without a source. In 2011 Mark Hughes reviewed five published instances of the claim for the Journal of Change Management and concluded there is "no valid and reliable empirical evidence to support such a narrative."
The closest thing to a measured 70% is BCG's 2020 study of 825 executives and 70 client programs. It found 30% of programs met or exceeded their target value, 44% created some value but missed targets, and 26% created little value. The "70% fail" headline adds the last two groups together. And it measured broad transformation programs, not the replacement of a specific system.
What the better studies say
Four sources are worth your time; each measures something different.
- Standish Group, CHAOS Report 2015. Across more than 25,000 software projects from 2011 to 2015, 29% were on time, on budget and on target, 52% were challenged and 19% failed. Size dominated: 61% of small projects succeeded against 6% of "grand" ones, and large, complex projects succeeded 2% of the time. Rebuilds from scratch succeeded 22–23% of the time; what Standish calls "modernization in place" succeeded 53% of the time.
- McKinsey and Oxford, 2012. A study of more than 5,400 IT projects found that large projects (over $15 million) ran 45% over budget, 7% over time and delivered 56% less value than predicted. Some 17% went badly enough to threaten the company's survival. Every additional year of planned duration added 15% to the cost overrun.
- Flyvbjerg and Budzier, Harvard Business Review, 2011. In 1,471 IT projects the average cost overrun was 27%, which sounds survivable. But one project in six was a "black swan" with an average overrun of 200% and a schedule slip of almost 70%. The average hides the tail, and the tail is what ends careers.
- Gartner, 2024. Its survey of 3,186 CIOs found that only 48% of digital initiatives meet or exceed their business outcome targets. A top cohort where CIOs and business leaders co-own delivery reached 71%.
The caveats
Read these numbers with care. Eveleens and Verhoef, writing in IEEE Software in 2010, applied Standish's definitions to 1,211 real projects and found they "didn't reflect the reality of the case studies at all", mainly because Standish scores success as estimate accuracy in one direction only, so a team that pads its estimates looks successful. Standish also changed its definition in 2015, moving the success rate seven points. The McKinsey and Oxford database leans toward large projects, and the Gartner and BCG figures are executive surveys, which measure perception as much as outcome.
What survives the caveats is consistent. Big and long is dangerous. Outliers are common. And the benefit side, not the cost side, is where most of the shortfall sits.
Why modernization projects miss
The studies describe the pattern. In our experience of 200+ projects at Thinslices, five causes account for most of it.
Scope that covers the whole system
"We'll replace it all in eighteen months" turns a bounded task into the grand project that succeeds 6% of the time. Every workflow depends on every other, and nothing releases until everything is ready.
Lost knowledge
Joel Spolsky's 2000 essay on the Netscape rewrite, Things You Should Never Do, made the point that old code is ugly because it carries years of bug fixes, each one a rule somebody learned the hard way. A rebuild that starts from a requirements document rather than the running system rediscovers those rules one production incident at a time.
Big-bang cutovers
Martin Fowler's strangler fig note describes the "simple replacement" plan: build a new system that does what the old one does, then switch. He has seen it "go down in flames most of the time," because existing behavior is hard to pin down and users can't wait years for new features. The Levi Strauss case in the Flyvbjerg and Budzier article is the extreme: a project budgeted under $5 million closed three US distribution centers for a week at switchover and ended in a $192.5 million charge.
No baseline
McKinsey's "56% less value than predicted" is only measurable because someone wrote down the prediction. Most modernization plans never record how long a change takes today, what the system costs to run, or how many defects reach customers. Without that, the project is judged on feel, and feel favors whoever is in the room.
Optimism bias
Lovallo and Kahneman's 2003 article Delusions of Success explains why estimates are wrong in the same direction every time. Planners take the "inside view," reasoning from their own plan, and ignore the "outside view" of how similar projects turned out. Their fix, reference-class forecasting, is to look up the distribution for comparable projects first. The numbers above are that reference class.
What the projects that landed did differently
The same data points the other way. Small projects in the CHAOS 2015 table succeeded ten times as often as grand ones. In the McKinsey study, a health care provider about to spend $1 billion over eight years was stopped at the green-light stage because the plan was twice as large and twice as long as anything comparable. Three patterns recur.
- Smaller scope. One workflow or one customer cohort at a time, released and measured before the next starts. Fowler's version: build around the edges of the old system and move behavior across in pieces, so "investment and returns occur gradually and visibly."
- Evidence before build. Record what the current system does, from traffic, jobs, data and reports, and have the business owners approve that record before anyone writes new code. A publisher's subscription system we worked on carried pricing rules that lived only in a database procedure and one person's memory. Observing the running system found them. The original plan would have found them in production.
- Measured pilots. Agree a baseline, run a bounded pilot, compare. The McKinsey and Oxford study found black-swan risk was driven largely by how long a project ran before going live. A pilot that goes live in months, while the current system keeps serving, is the cheapest way to shorten that exposure.
Score your own plan
One point for each statement that is true today, not planned.
- The first release is one workflow or one customer cohort, not the whole system.
- Someone has recorded how the current system behaves, from evidence, before build starts.
- The business owners have reviewed and approved that record.
- A baseline exists in writing: cost to run, time to make a change, defects reaching customers.
- The estimate has been checked against comparable projects, not only built bottom-up.
- The current system keeps serving customers until the replacement is proven against it.
- Each move of traffic has acceptance criteria and a rollback that has been tested.
- Spend and progress are reported each period, and there is a written stop condition.
Seven or eight puts you in the minority that lands. Four to six means fix the gaps before you fund the work. Below four, the plan is a big-bang rewrite with a newer name, and you've just read its reference class.
Where Fabrica stands
Fabrica's CLEAR method is built around the items on that list. Capture records how the running system behaves; Lock has your owners approve a versioned System Specification; Attest tests the rebuilt workflow against the current system before any traffic moves; Release moves one workflow or cohort at a time with acceptance criteria and a rollback that is a routing change. An assessment and pilot measures the result against a baseline you agree, which is how we'd want to be judged.
The odds are not fixed
The failure statistics are real, but they describe a way of running projects, not a law of nature. The same databases that produce the headline numbers show which projects beat them: small, evidence-led, measured, and released while the old system kept running. Don't argue with the odds. Change the reference class your project belongs to.
Questions to ask before you approve a modernization plan:
- Which study is that failure figure from, and what did it actually measure?
- What is the smallest first release that would prove the approach on our system?
- What is our baseline, who signed it, and when was it measured?
- How would we know in month three that we should stop?
- On the day traffic moves, what do customers see, and how do we move it back?
