Eval scores improve while user satisfaction is flat or declining
Internal AI evaluation scores keep getting better, but real users do not report better outcomes.
- Where you see this
prompt optimizationmodel selectionRAG quality tuningAI assistant rollout
- Not necessarily a problem when
- the improvement is on a narrow internal capability that has not yet shipped
- Often mistaken for
- the eval says it is better
- Time horizon
- near-term
- Best placed to act
AI feature ownerML engineerproduct owner
The signal
What you would actually notice
The team may be optimizing against a proxy that no longer represents real task success.
Field observation
Reports show improving eval results while support tickets, adoption, completion rates, or user feedback do not improve.
Also observed
- The eval is up five points, but complaints are unchanged.
- The model is better, users just do not see it yet.
- The bad cases are outside our eval scope.
Primary reading
What it usually indicates
Most likely underlying patterns when this signal shows up. Not a diagnosis, a starting hypothesis.
Usually indicates
Most likely underlying patterns when this signal shows up.
- eval goodhart
- weak evaluation discipline
- stale task coverage
- synthetic evaluation overfit
Stakes
Why it matters
The team may be optimizing against a proxy that no longer represents real task success.
Inspection
What to check next
Deliberate steps to confirm or disconfirm the primary reading above. Not a checklist. An order of inspection.
- eval set freshness
- production failure samples
- task-level score breakdown
- user outcome metrics
- prompt or model change history
Diagnostic questions
Questions to ask the team, or yourself, before concluding anything.
- Which user outcome should this score predict?
- Where do production failures appear in the eval set?
- Which task slice improved?
- Has the team optimized prompts directly against the eval?
Progression
Under the signal
Where this pattern tends to come from, what's holding it up, and where it goes if nothing changes.
Leading indicators
What tends to show up first.
- aggregate eval scores are discussed without task slices
- failed user sessions are not added to evaluation
- the eval set has not changed while the product did
Common root causes
What is usually sitting under the signal.
- eval set overfitting
- stale task assumptions
- metric pressure
- weak production feedback loop
Likely consequences
What happens if nothing changes.
- false launch confidence
- eval goodhart
- silent model drift
- loss of trust in evaluation
Look-alikes
Not what it looks like
Patterns that can be mistaken for this signal, and 'fix' attempts that make it worse.
- the eval says it is better
- users need time to notice
- support feedback is noisy
Anti-patterns when responding
Responses that feel sensible and usually make the underlying pattern worse.
- celebrating aggregate score improvement without user evidence
- changing prompts until the eval passes
- discarding user complaints as anecdotal
Context
Context and ownership
Where this signal surfaces, who sees it first, who can actually act, and how much runway there usually is before escalation.
Where it shows up
- prompt optimization
- model selection
- RAG quality tuning
- AI assistant rollout
Who sees it first
Before it escalates.
- support lead
- product manager
- data analyst
Who can move on it
Not always the same as who notices it.
- AI feature owner
- ML engineer
- product owner
near-term
How much runway there usually is before the signal hardens into the underlying pattern.
AI impact
AI effects on this signal
How AI-assisted and AI-driven workflows tend to amplify or hide this signal.
AI amplifies
Ways AI tooling tends to make this signal louder or more common.
- AI can tune prompts quickly against the same eval set until scores improve without behavior becoming more useful.
AI masks
Ways AI tooling tends to hide this signal, so it keeps growing under the surface.
- AI-generated analysis can rationalize score improvements even when outcome evidence is weak.
Relationships
Connected signals
Related failure modes, decisions behind the signal, response playbooks, and neighboring red flags.