Most AI systems look effortless from the outside. A model, some data, a prediction, delivered with confidence and a clean interface. That polish is (often) deliberate, and it hides how much sits beneath the surface. By now, most teams building in production are catching on: the demo is the easy part.
In 2015, a group of Google engineers published a paper that has aged unusually well. “Hidden Technical Debt in Machine Learning Systems” made an observation that was almost provocative in its simplicity. In a real-world machine learning system, only a small fraction of the code does the actual learning. The vast majority is the machinery around it: configuration, data collection, feature extraction, data verification, resource management, serving infrastructure, and monitoring. The model is the visible tip of the iceberg. Everything that keeps it useful in production lives underwater.
Nearly a decade later, that ratio has not changed. If anything, it has become more relevant, because the ambition around AI has grown far faster than the discipline underneath it. Organisations now want models embedded in decisions, forecasts, and daily operations. The 5% gets the attention and the budget. The 95% is where projects quietly stall or disappear altogether.
Why the debt compounds
Technical debt in AI is not a single problem. It is a set of failure modes that reinforce each other, and most of them are invisible until they become expensive.
The first is what the Google authors called the CACE principle: Changing Anything Changes Everything. Machine learning systems entangle their inputs. Adjust one feature, add one data source, or change one threshold, and the effect ripples across the whole model, altering weights and importance in ways that are hard to predict. There are no clean interfaces between components the way there are in traditional software. This makes every change riskier than it looks, and it makes systems progressively harder to reason about as they grow.
The second is that data dependencies cost more than code dependencies. Code dependencies are at least visible; a compiler or a linter will complain. Data dependencies are silent. An upstream table changes format, a source system is upgraded, a definition drifts, and model performance degrades without anything obviously breaking. These dependencies are difficult to track and even harder to test, and they accumulate faster than teams expect.
The third is the slow tangle of pipeline jungles and correction cascades. Data preparation rarely arrives fully formed. It grows organically: a scrape here, a join there, a quick fix layered on top of an earlier quick fix. Over time this becomes a jungle of transformations that no single person fully understands, prone to failure and painful to change. Corrections cascade, because fixing one downstream issue often requires touching several upstream steps that were never designed to be touched.
Around all of this sit two quieter forms of debt: configuration that nobody treats as code, versioned and reviewed like the rest of the system, and features that were once useful but are now dead weight, still consuming compute and complexity while adding nothing. Neither is dramatic on its own. Together they make a system heavier, slower, and more fragile every quarter.
Paying it down is an operational discipline
The uncomfortable truth is that none of this is solved by a better model. It is solved by operational discipline. The same paper that named the problem also pointed at the remedies, and they are strikingly unglamorous: standardise configurations and treat them as code, eliminate underutilised features rather than tolerating them, monitor for prediction bias and drift so you notice degradation before your users do, and design deliberately to keep pipelines simple instead of letting them sprawl.
This is where the conversation has to move away from the model and toward the data stack. An enterprise that wants to use AI well is really asking a different question than “which model should we use.” The real question is whether the data foundation underneath can support a model at all without collapsing under its own maintenance burden. AI-readiness is not a property of the algorithm. It is a property of the stack: the pipelines, the definitions, the governance, and the operating model that feed it.
What this looks like in practice
Consider a transformation we worked on with a global industrial services company whose challenge was never a shortage of data. It was the opposite.
Over years, data had accumulated everywhere: inside operational systems, regional ERP instances, legacy warehouses, and countless reports built to solve local problems quickly. Each solution worked in isolation. Together they formed a brittle landscape of technical debt. Reports contradicted one another. KPI definitions varied by country. A single change to business logic meant manual fixes across dozens of dashboards. Teams spent more time maintaining pipelines than producing insight, and a handful of experts who “knew how things worked” had become a bottleneck the whole organisation depended on.
The instinct in that situation is to give more people access to more tools. That only accelerates the chaos. What the company needed was a reset, not just of technology but of how analytics was built, governed, and used.
We proposed starting with structure rather than dashboards, on one principle: discipline at the core, flexibility at the edge. True self-service can only exist on a stable foundation. That meant shared data models, consistent definitions, and clear ownership before anyone built anything new. The transformation was staged deliberately. Legacy assets were inventoried and assessed, high-value reports were prioritised rather than blindly migrated, and every migration was treated as a chance to simplify and consolidate instead of replicating old patterns in a new tool.
The most important shift was treating the platform as engineered infrastructure rather than a collection of outputs. Operational data flowed through standardised ingestion pipelines, was enriched and standardised centrally, and was then exposed through shared semantic models. Business logic stopped living inside individual reports and moved into governed, documented datasets that reports simply consumed. That single act of decoupling, separating dataset creation from report creation, removed a major source of debt. Users could build and adapt freely without re-implementing transformations or redefining KPIs each time.
Migration was handled in parallel with the legacy environment so teams could validate results and build confidence before anything was switched off. Each successful migration retired another legacy report, and the landscape steadily became smaller, cleaner, and easier to manage. Governance, often seen as a brake, became an accelerator: a pragmatic model and a Center of Excellence made it easy to do the right thing, coaching business users and continuously reducing debt rather than policing access.
The targets that guided the work
From the outset, we aligned with the client on measurable design goals. Not marketing numbers, but practical targets used to manage scope and trade-offs:
- No more than three certified semantic models per core domain, to prevent model sprawl and enforce reuse.
- 70 to 80 percent report reuse through shared datasets, shifting logic out of individual reports.
- A 30 to 50 percent reduction in duplicated transformation logic, by centralising data preparation.
- A single KPI definition per metric at enterprise level, with controlled local extensions rather than parallel definitions.
- Dev, Test, and Prod deployment for all critical assets, to reduce incidents and enable controlled releases.
- A Center of Excellence response time within five business days for enablement requests, so governance supported speed.
- A parallel-run window of 30 to 90 days per migrated solution, to de-risk the switch.
These constraints shaped architecture, sequencing, and governance, and kept the focus on long-term sustainability rather than short-term wins.
The result was more than reporting efficiency. By removing the hidden debt and modernising the foundation, the company created clean, governed, well-modelled data that was genuinely ready for advanced analytics, forecasting, and automation. What began as a modernisation effort became a stepping stone toward AI-driven decision-making. As their Data & Analytics Leader put it:
"For years, our BI environment kept growing, but our ability to trust and scale insights didn't. Every change added more complexity. Intellerts helped us step back and redesign analytics as a platform, instead of a collection of reports. Today, business teams can work independently without breaking consistency, and IT is no longer trapped maintaining legacy logic. The biggest change is not the tools. It's the confidence that our data and metrics mean the same thing everywhere."
AI Debt: The Next Layer Beneath the Iceberg
Technical debt is no longer the only liability hidden beneath AI initiatives. Gartner has introduced the concept of AI Debt: the accumulated cost of decisions that prioritise rapid AI delivery over long-term sustainability, creating future burdens such as rework, complexity, operational risk, and missed value.
In many ways, AI debt is what happens when technical debt meets enterprise AI ambition.
The same fragmented data sources, duplicated business logic, inconsistent KPI definitions, and unmanaged dependencies that make analytics difficult also make AI harder to scale. Organisations often discover that their biggest obstacle is not model performance, but the debt embedded in the underlying data stack. Every new copilot, forecasting model, or AI agent built on top of inconsistent foundations inherits and amplifies that complexity.
This is why AI readiness is fundamentally a data and operating model challenge. Before asking whether a model is powerful enough, organisations should ask whether their data platform, governance model, and semantic layer are robust enough to support AI without creating another generation of debt.
The companies that scale AI successfully are rarely the ones with the most advanced models. They are the ones who have already paid down enough technical and data debt to give those models a stable foundation.
The point underneath the point
This reflects something we believe firmly: AI tools are not magic, and outcomes are engineered. Without strong data foundations, governance, and operating models, AI cannot deliver value, no matter how capable the model. The debt hidden beneath a data stack is precisely what determines whether that stack is ready for AI or quietly working against it.
Most organisations carry more of this debt than they can see. That is the nature of it.
If the iceberg looks familiar, our short video walks through the framework in a few minutes, and it’s a good place to start. If you suspect your own data stack is carrying hidden technical debt, we’re always open to a conversation with one of our experts to help you find out where it sits and what it would take to pay it down.
