Conventional software fails loudly. AI systems fail quietly. A broken API throws a 500 and pages someone; a degraded agent keeps returning confident, well-formatted, subtly wrong answers while every dashboard stays green. Uptime is 100%. Latency is fine. Error rate is zero. And quality has been sliding for five weeks.
This is the part of AI work almost nobody prices in, and it's the reason a system that launched well can be worthless by the following quarter. Launch is not the finish line — it's the point at which the system starts drifting away from the conditions it was built for.
The five ways a live AI system degrades
| Failure | What changed | How it shows up |
|---|---|---|
| Model drift | The vendor retuned or deprecated the model | Behaviour shifts overnight, no deploy on your side |
| Data drift | Your inputs stopped looking like your test set | Accuracy decays gradually, worst on new cases |
| Workflow drift | The humans changed the process | The agent is now correct about a stale reality |
| Integration rot | A downstream API changed shape | Silent partial failures on one path |
| Prompt entropy | Everyone edited the prompt a little | Nobody can say what current behaviour is or why |
Model drift is the one people underestimate. You do not control the schedule — vendors retune, deprecate, and ship behaviour changes when it suits them, and a system pinned to nothing inherits every change. Without versioned configuration and a rollback path, "it used to work" becomes an archaeology project.
Data drift is the slowest and most expensive. The system was tuned against the cases you had. Your customers change, your product changes, a new segment arrives, and the live distribution walks away from the test set. Nothing breaks. It just gets worse, in a way only measurement catches.
Workflow drift is the funniest and the most human. Ops changed a step, added a queue, renamed a status — and the agent is now faithfully executing last quarter's process. It isn't wrong about anything except reality.
Why nobody notices for weeks
Because the signals people watch are the wrong ones. Uptime, latency, and error rate all describe whether the system *ran* — none of them describe whether it was *right*. Quality is not a property you can infer from infrastructure metrics, and it's the only property that matters here.
Then the volume hides it. If an agent handles 4,000 cases a month and quality drops from 94% to 88%, that's 240 additional wrong outcomes — invisible in aggregate, distributed across the exact customers least likely to write in about it. You typically find out through a downstream complaint, weeks late, and then have to reconstruct what happened without traces because nobody was recording them.
This is also why the adoption gap persists: 91% of mid-market firms use generative AI, but only 25% have it integrated into core operations (RSM, 2025). Integrating into core operations means someone has to be accountable for a live system on a Tuesday — and that job has to exist before it can be done.
What "operating" an AI system actually means
It's a real job with a real definition, not a support contract. Concretely:
- Eval regressions on a schedule. The eval suite runs continuously, not once at launch, so quality movement is a number someone sees rather than a thing a customer reports.
- Trace review. Someone reads real runs — especially the escalations and the low-confidence ones — because that's where drift shows up first.
- Model migration. Tracking deprecations, testing the replacement against the eval set *before* the forced cutover, and keeping a rollback path warm.
- Prompt and config version control. Changes are reviewed and versioned, so current behaviour is always attributable to a specific decision.
- Cost and latency watch. Spend per transaction drifts with usage patterns and model choices, usually upward and usually unnoticed.
- Workflow liaison. Somebody who knows when ops changed the process, because ops will not think to tell the AI vendor.
That last one is why we do this forward-deployed rather than through a ticket queue. The signal that a workflow changed doesn't arrive as a bug report — it arrives as a hallway conversation, and you have to be close enough to the work to hear it. The layers being watched are the ones in the production agent stack; operating is what keeps them true after launch.
What it costs, and what skipping it costs
Ours is $3,000–$20,000 a month for managed AI operations, scaling with how many systems are under management — one system at the light end, five or more production agent platforms at the heavy end. Whether you buy that or staff it internally matters less than whether the function exists and someone owns it.
The alternative isn't free, it's just unbilled. It shows up as a quarter of degraded output nobody measured, a forced model migration done under deadline, and eventually a rebuild of something that worked at launch — which is how a large share of the 40%+ of agentic-AI projects Gartner expects to be canceled by 2027 (Gartner, 2025) will actually end. Not with a failure. With a slow decline nobody was watching.
