AI agent operations runbook
53 steps to keep an AI agent working after launch. Each one says when it is done, so you can see what your team has and what is missing.
The runbook as a spreadsheet
53 rows, one per item, with blank Status, Owner and Notes columns to fill in. Opens in Excel, Google Sheets or Numbers. Free, and no email needed.
Work through it once per agent, then keep it current.
Make one copy for each agent in production. Mark each step Done, Partly or Not started, and give every gap an owner and a date.
Start with scope, versions and evals. Without a scored baseline, the alerts and rollbacks have nothing to measure against. Then add monitoring, tracing and the incident steps.
For what each entry is for, read what an AI operations runbook actually contains. For the tools that capture traces, see AI agent observability tools.
Ten areas, from scope to the quarterly review.
The same steps as the CSV, in the same order.
Scope and limits
The first question in any incident is what the worst outcome could have been. Answer it before the incident.
- 01Write down what the agent does, in two or three sentences.Someone who did not build it can read the paragraph and say what the agent is for.
- 02List every system the agent can read from and write to.Each system is named with the permission the agent has: read, write or both.
- 03List the actions the agent must never take.A short list, such as issuing refunds or deleting records, that is also enforced in code or permissions.
- 04Mark the actions that need a person to approve.Each one names who approves it and where the approval happens.
- 05Write the business fallback for when the agent is off.The manual process the team uses, and who runs it.
Inventory and versions
When behaviour changes, you need to know exactly what was running. Pin everything.
- 06Record the model name and exact version for every call.The version string is in config, not a model alias that can change underneath you.
- 07Keep prompts in version control.Every prompt change is a commit with an author and a date.
- 08Record the tools and their versions.Each tool or integration the agent calls, with its API version.
- 09Record the retrieval setup if the agent uses documents.Which index, which embedding model and when it was last rebuilt.
- 10Tag each release.One tag covers prompt, model, tools and config together, so you can name what was live on any day.
Evals and baseline
Without a scored baseline, "is it worse?" is an opinion. The eval suite turns it into a number.
- 11Build a test set from real cases.Inputs with the correct outcome, including the hard and unusual ones, stored with the code.
- 12Add a case every time something fails in production.The failure becomes a test that runs on every release.
- 13Score the current release and record it as the baseline.The scores are written down with the release tag and the date.
- 14Run the suite before every release.A release that scores below the baseline does not go out without a written reason.
- 15Run the suite on a schedule, even with no release.It runs at least weekly, so a vendor-side change shows up as a drop.
- 16Keep the last several results side by side.Drift is visible as a trend, not a single reading.
Monitoring and alerts
Infrastructure dashboards can stay green while answers get worse. Alert on quality, not just uptime.
- 17Set an alert on error rate.A threshold and who gets paged when it is crossed.
- 18Set an alert on latency.A threshold on slow responses, measured where the user waits.
- 19Set a quality threshold.The eval score, or a sampled review score, below which someone is woken up. Decided in advance.
- 20Watch for unusual tool use.An alert when the agent calls a tool far more or less often than normal, or calls one it rarely uses.
- 21Watch the rate of hand-offs to people.A sudden rise or fall in escalations is a signal worth checking.
- 22Test that alerts reach a person.Trigger each alert once on purpose and confirm the right person got it.
Tracing and logs
When one case goes wrong, you need to see exactly what the agent saw and did.
- 23Capture a trace for every run.Input, retrieved context, each tool call in order, the output, the cost and the time taken.
- 24Link each trace to a user or ticket ID.Support can find the trace for a complaint without asking engineering.
- 25Remove or mask personal data the trace does not need.Logging follows the same rules as the rest of your data.
- 26Set how long traces are kept.A written retention period that matches your privacy policy.
- 27Write one worked example of reading a trace.A real failure, walked through from input to output, kept in the runbook.
Incident response
Write the steps while you are calm. During an incident, people follow the page, not their memory.
- 28Define severity levels for this agent.What counts as minor, major and critical, with an example of each.
- 29Write the first five minutes.Confirm the problem, check recent releases and vendor status, decide whether to switch off.
- 30Name who can switch the agent off.At least two people, and the exact way to do it.
- 31Have a switch that turns the agent off without a deploy.A feature flag or config change that sends traffic to the fallback in minutes.
- 32Write who is told, and when.Support, the business owner and, if customers are affected, whoever speaks to them.
- 33Record what the agent did during the incident.The affected cases are listed from traces so they can be checked and put right.
- 34Hold a short review after every major incident.What happened, why, and the test or alert that would have caught it, written within a week.
Rollback and release
Rolling back should be a deploy, not an investigation.
- 35Know the last good release.Its tag is written in the runbook and updated after each release.
- 36Write the exact rollback steps.The commands or clicks, in order, tested at least once.
- 37Roll back prompt, model and config together.One action returns all three to the last good release.
- 38Release changes to a small share of traffic first.A canary or staged rollout, watched against the alerts before full release.
- 39Keep a release log.Every release, who shipped it, what changed and its eval score.
Vendor and model changes
Model vendors retire and change models on their own schedule. Find out weeks ahead, not on the day.
- 40List every model and vendor the agent depends on.Including models used for embeddings, moderation or routing.
- 41Record the announced retirement date for each model.Taken from the vendor's own deprecation page, with the link.
- 42Subscribe a named person to each vendor's change notices.Status page, changelog and deprecation emails go to someone who reads them.
- 43Test a replacement model before the old one is retired.The eval suite runs on the new model, and the result is recorded.
- 44Have a second provider ready for critical paths.Or a written decision that the agent waits during an outage, and what the business does meanwhile.
Cost and limits
A loop or a traffic spike can run up a bill overnight. Put limits in place before it happens.
- 45Track cost per run and per day.A daily figure someone looks at, with the cost per run from traces.
- 46Set a budget alert.An alert when daily spend passes a set amount.
- 47Cap steps and tokens per run.A run that loops stops at a set limit and hands off to a person.
- 48Know your rate limits.The vendor's limits for your account, and what the agent does when it hits them.
Ownership and review
A runbook nobody owns goes stale. Give it a name and a date.
- 49Name the owner of the agent.One person, with a deputy for when they are away.
- 50Name the business owner.The person who decides what good looks like and signs off on changes in behaviour.
- 51Review the runbook after every release.Versions, last good release and contacts are current.
- 52Test the runbook with someone who did not build the agent.They can find the cause of a planted problem using only the runbook.
- 53Review quality and cost every quarter.Eval trend, incidents, spend and the plan for the next quarter, in one short meeting.
Questions about the runbook
What goes in an AI agent operations runbook?
The steps a team needs to keep an agent working after launch. This one has 53 steps in 10 areas: scope and limits, versions, evals and a baseline, monitoring and alerts, tracing, incident response, rollback, vendor and model changes, cost limits, and ownership.
How is it different from a normal service runbook?
An agent can fail while every server is healthy: the answers get worse, a tool is called too often, or the model changes underneath you. So the runbook adds a quality threshold from evals, pinned model and prompt versions, and a watch on vendor retirement dates.
Who should own it?
The engineer who owns the agent, with a deputy, and a business owner who decides what good looks like. Test it by asking someone who did not build the agent to find a planted problem using only the runbook.
We run the evals, alerts, fixes and model changes for you.
Managed AI Operations starts at $3,000/mo for 1 system, with a monthly improvement plan and a quarterly review.