Monitor model behavior in production
Monitor model and system behavior through task-level production signals, drift checks, review outcomes, and rollback triggers.
- Situation
- An AI feature is live or near launch, and the team needs to notice behavior change before users or support discover it informally.
- Goal
- Detect harmful behavior changes early enough to intervene safely.
- Do not use when
- the feature is not production-facing
- Primary owner
- AI feature owner
- Roles involved
AI feature ownerML engineerproduct manageroperations ownersupport leadrisk owner
Context
The situation
Deciding whether to reach for this playbook: when it fits, and when it doesn't.
Use when
Conditions where this playbook is the right tool.
- Model or prompt behavior can change after launch
- Human review is part of the safety model
- Support sees patterns before metrics do
- Offline evals are not enough to represent production behavior
Stakes
Why this matters
What this playbook protects against, and why skipping or half-running it tends to be expensive.
AI quality can degrade without a deployment event. Production monitoring keeps behavior trust from depending on anecdotes, luck, or user complaints.
Quality bar
What good looks like
The observable qualities of a team or system that is actually doing this well. Not just going through the motions.
Signs of the playbook done well
- Production signals are tied to tasks
- Known failure classes are monitored
- Human review catches are measured
- Rollback or containment triggers exist
- Monitoring feeds the eval harness
Preparation
Before you start
What you need available and true before running the procedure. Skipping this is the most common reason playbooks fail.
Inputs
Material you'll want to gather first.
- Task inventory
- Offline eval results
- Production events
- Human review logs
- Support feedback
- Rollout plan
Prerequisites
Conditions that should be true for this to work.
- Instrumented production behavior
- Named action owner for alerts
- Rollback or containment path
- Task-grounded baseline
Procedure
The procedure
Each step carries its purpose (why it exists), its actions (what you do), and its outputs (what you produce). Read the purpose. It's what keeps the step from degenerating into checklist theatre.
Choose task-level production signals
Avoid monitoring only generic usage or satisfaction.
Actions
- Map launch-critical tasks to observable events
- Identify user, support, and operational signals
- Separate quality, safety, and adoption signals
Outputs
- Production signal map
- Task-level monitoring plan
Define drift and failure triggers
Make response conditions clear before drift appears.
Actions
- Define expected baseline behavior
- Set thresholds or review triggers for known failure classes
- Name containment, rollback, or escalation actions
Outputs
- Drift trigger list
- Response playbook
Monitor human review quality
Prevent human-in-the-loop controls from decaying silently.
Actions
- Track what human reviewers catch
- Review skipped or rushed approvals
- Sample approved outputs for judgment quality
Outputs
- Review quality signals
- Human-review catch report
Feed production learning back
Keep offline evidence aligned with live behavior.
Actions
- Add production failures to eval cases
- Review monitoring gaps after incidents
- Update alerts and evals after behavior changes
Outputs
- Production-to-eval loop
- Monitoring gap log
Judgment
Judgment calls and pitfalls
The places where execution actually diverges: decisions that need thought, questions worth asking, and mistakes that recur regardless of good intent.
Decision points
Moments where judgment and trade-offs matter more than procedure.
- Which signals imply user harm or trust loss?
- What behavior triggers rollback?
- Which review failures indicate human-in-the-loop decay?
- Who is paged or notified for model behavior drift?
Questions worth asking
Prompts to use on yourself, the team, or an AI assistant while running the procedure.
- Which production signals would reveal model drift for this task?
- What user-visible failures are not represented in monitoring?
- What should happen if human review catches fewer issues over time?
Common mistakes
Patterns that surface across teams running this playbook.
- Monitoring usage but not task quality
- Creating alerts with no response owner
- Assuming offline evals cover production drift
- Ignoring human review catch rates
Warning signs you are doing it wrong
Signals that the playbook is being executed but not landing.
- Support sees a pattern before metrics do
- Quality feels different but nobody can prove it
- Review is described as a formality
- Model drift is noticed informally, not measured
Outcomes
Outcomes and signals
What should exist after the playbook runs, how you'll know it worked, and what to watch for over time.
Artifacts to produce
Durable outputs the playbook should leave behind.
- Production signal map
- Drift trigger list
- Response playbook
- Review quality report
- Monitoring gap log
Success signals
Observable changes that mean the playbook landed.
- Behavior changes are detected before broad user impact
- Alerts lead to action
- Human review catches are visible
- Production failures improve offline evaluation
Follow-up actions
Moves that keep the playbook's effects compounding after it finishes.
- Review monitoring after each model or prompt change
- Sample production behavior regularly
- Rehearse rollback or containment paths
Metrics or signals to watch
Longer-horizon indicators that the underlying problem is receding.
- Task success by segment
- Known failure recurrence
- Human review catch rate
- Support pattern reports
- Rollback or containment events
AI impact
AI effects on this playbook
How AI-assisted and AI-driven workflows help execution, and the ways they can make it worse.
AI can help with
Where AI tooling genuinely reduces the cost of running this playbook well.
- Clustering production failures
- Summarizing drift patterns
- Drafting monitoring checks
- Comparing live behavior against baseline examples
AI can make worse by
Distortions AI introduces that make the underlying problem harder to see.
- Summarizing drift too smoothly
- Generating dashboards without response design
- Masking rare high-severity failures behind averages
Relationships
Connected playbooks
Failure modes this playbook tends to address, decisions behind the situation, red flags that motivate running it, and neighboring playbooks.