A runbook exists so that the person holding the pager at 7am doesn't have to reconstruct your architecture from first principles. For conventional software that's well-trodden. For an AI system it's harder, because the failures are quieter, the causes are often outside your codebase, and "is it broken?" is a question with a numeric answer rather than a yes or no.
Here's what actually needs to be in one. Seven entries — each answering a question somebody will ask at an inconvenient hour.
1. What this system does, and what it must never do
Two paragraphs at the top: the workflow it owns, the systems it can write to, and the explicit list of actions it is not permitted to take. The negative list matters more than the positive one during an incident, because the first question in any AI incident is *what's the worst this could have done?* and nobody should be inferring the answer from source code under pressure.
2. The eval suite and its schedule
Where the test set lives, how to run it, what the current baseline scores are, and how often it runs unattended. Include the last several runs, so drift is visible as a trend rather than a single reading.
This is the entry that makes the rest possible. Without a scored baseline, "is it degraded?" is an opinion — which is why we treat the eval suite as the first gate, not a nice-to-have.
3. Alert thresholds — including the quality one
Latency and error rate, obviously. But the entry that distinguishes an AI runbook is a quality threshold: the eval score below which someone gets woken up. Most teams never set one, which is precisely why degradation runs for weeks unnoticed while infrastructure dashboards stay green.
Set the number in advance, when you're calm. Deciding what counts as "too degraded" during an incident guarantees you'll rationalise upward.
4. How to read a trace
For any single run: where the traces live, and how to reconstruct what happened — the input, the retrieved context, the tool calls in order, the output, the cost, the latency. Include one worked example of a real failure walked through end to end.
That worked example is the highest-value paragraph in the document. It's the difference between having observability and knowing how to use it at speed.
5. Rollback: what to revert, and to what
Which prompt versions, model versions, and configuration are currently pinned; where the last known-good state is; and the literal command sequence to return to it. Rolling back should be a deploy, not an investigation.
Include the model version explicitly. When a vendor ships a behaviour change, "revert to yesterday's config" is meaningless if the config didn't pin the model.
6. The vendor-change watch
Which model versions you depend on, their announced deprecation dates, where those announcements get posted, and who is subscribed. Model vendors deprecate and retune on their schedule, not yours, and a forced migration discovered at the cutover is an emergency; discovered eight weeks out it's a sprint task.
7. Ownership and escalation
Who owns this system by name. Who to call when they're unavailable. Who can authorise turning it off, and — importantly — what the business fallback is while it's off. If there is no manual process to fall back to, that's worth knowing before the day you need one.
The whole thing, on one page
| Entry | The question it answers at 7am |
|---|---|
| Scope and prohibitions | What is the worst this could have done? |
| Eval suite and baseline | Is it actually degraded, or does it just feel wrong? |
| Alert thresholds | Should someone have been woken up sooner? |
| Trace reading | What did it do on this specific case? |
| Rollback | How do we stop the bleeding right now? |
| Vendor-change watch | Did something change underneath us? |
| Ownership and escalation | Who decides, and what do we do meanwhile? |
What a bad runbook looks like
Three failure modes, all common. Architecture diagrams instead of procedures — beautiful, and useless at 7am; a runbook is a set of instructions, not an explanation. No numbers — "monitor for degradation" is not a threshold. And written by someone who has never been paged, which produces a document describing the system as designed rather than as it fails.
The test is simple: hand it to a competent engineer who didn't build the system and ask them to diagnose a degradation. If they can't get to a hypothesis in fifteen minutes, it isn't a runbook yet.
Who writes it, and when
It gets written during the build, not after — the knowledge is freshest then, and requiring it forces the observability to actually exist. In our Build phase the runbook and the eval suite ship with the system as part of the deliverable, alongside the code, because a production system handed over without them isn't finished.
If you'd rather someone else hold the pager, that's what Managed AI Operations is — continuous evals, drift and quality monitoring, prompt and model versioning, vendor-change response, and a quarterly roadmap against your KPIs, from $3,000 a month for a single system. Either way, the runbook has to exist. The only question is whose desk it sits on.
