The Regression Detector

Help me detect when an agent regresses.

Agent: [DESCRIBE]
What changed: [MODEL VERSION / PROMPT / TOOLS / NOTHING —
BEHAVIOUR JUST DRIFTED]
Symptoms: [DESCRIBE, OR "trying to detect proactively"]

Produce:
1. Metrics that would show a regression before users complain
2. For each metric: what a normal range looks like, and what
   change is significant
3. Behavioural signals beyond output quality: tool call count,
   run length, cost per run, escalation rate, retry rate
4. A canary approach: what to run continuously against known cases
5. How to attribute a regression to its cause
6. What to do when the model changed beneath you and nothing on
   your side did

That last one deserves attention. Model providers update models.
Behaviour changes without any change on your side. Produce:
7. What to pin and what to leave floating
8. How to test a new model version before adopting it
9. What in my prompts is likely to be version-sensitive
10. A migration checklist for when a model version changes