still trusting one good agent run? that proves it can, not that it will
vercel's evals guide:
- pass@k: any of k runs passes. fine when retries are cheap (coding + tests)
- pass^k: all k pass. use when a miss costs the user (orders, payments)
5/5 just means no failures seen yet
1
3
126
at 7:35 a.m. ET, GitHub said about 2–3% of merges were timing out, up from about 0.2% normally. it’s still investigating
githubstatus.com/incidents/2…
20
they are about to catch a skyscraper that just orbited around Earth with toothpicks
Watch Starship Flight 15
nitter.cf/i/broadcasts/2brajgZlO…
89
most llm-judge evals are a vibe check with a spreadsheet
anthropic's eval guide, ported to agent evals:
- blind hand-label a sample, check the judge agrees
- one bad turn fails the whole chat
- pick the metric before you look
- single-turn is saturated (94-98% on recent claudes)
7
3
207
switched models? your claude.md is still prompting the old one
run /doctor prompt-audit:
- reads claude.md, agents.md, skills
- flags old-model habits, dead paths, contradictions
- proposes diffs, edits nothing till you say so
it skips settings, so hooks are still your problem
people who started agents on opus 5.5 skipped the free distrust training the bad models gave the rest of us
steal it anyway:
- ask what it skipped, not what it did
- make it run the tests before it says done
- commit before every big ask
the magic is real. so is the rollback
53










