Pinned Tweet
Enjoy it. the window is closing faster than people think. the IPO wave hits in October. openai, the rest of the pack. that's when the subsidies get pulled. 6 months from today we're closing the last chapter of the cheap ai era. What's $200 a month right now turns into $20,000 when the discount disappears. right now you have 100x the capacity you'll have later, shared across every user hitting the same models. most people are spending it on toy prompts. spend like an empire while the tap is still open.
the reason AI teams don't touch their LLM costs is fear of a regression nobody can reproduce. That fear is based on a mental model of compression that doesn't apply to Anyray. @AnyrayHQ compresses your token inputs before the call, no code changes, no output degradation
People always ask how we optimize agent data without breaking the prompt cache.
One rule: hide unrelated data only where the provider hasn't cached yet. Then send that same version every turn.
127 passing tests hidden. The failure kept. Cache still warm.
Correction: dug deeper and it isn’t threads.
With a custom base URL, Claude Code leaves out the <total_tokens>… tokens left</total_tokens> budget notice from the system prompt. Add it back and parallel calls return (3/8 → 10/10 in replays; full sessions 2/5 → 5/5, ~$0.22 → $0.18).
Details: github.com/anthropics/claude…
A pricing agent. One competitor page. One hop later, a trap.
Humanbound ran 1,350 attempts. Undefended, four of five non-OpenAI models leaked a confidential unit cost every time — just from reading the web.
Hardened prompts mostly stopped the leak. Then the agent guessed. Across 27 model×guardrail combos: zero outcomes that were both safe and well-grounded.
The prompt isn't the boundary. Egress and provenance are. If security changes when you swap models, you don't have a control — you have a coin flip.
The failure conversation can be innocent. The memory that prepared it arrived earlier.
GhostWriter (Jul 2026): ~98% success planting an adversarial memory, ~60% average activation later. Microsoft also found 31 companies trying to poison assistant recommendations.
If your agent can act, "remembered past" is permission. Treat memory writes like untrusted tool output.
Most agent stacks add tracing after the first ugly 3am.
One builder flipped it: full turn / model-call / tool spans *before* writing the agent loop. Langfuse for humans. In-process trace for evals so a mapper bug can't fake a policy failure.
You debug what you can see. Start there.
Multi-agent bills hide a line item most dashboards don't name.
A recent breakdown (Total Cost of Agency) put memory injection around ~12% of billed cost — and the share climbs as the workflow gets deeper.
You pay for what the agent remembers, not just what it thinks.
Scar tissue isn't free.
Rogue agent vs hijacked agent are not the same failure.
Rogue = your loop went sideways.
Hijacked = someone else's prompt is living inside your permissions.
Same blast radius. Different forensics.
Most "agent went crazy" writeups are still mixing the two.
Every model launch, people google the token price.
Wrong number.
Agents don't buy tokens. They buy loops: retries, tool calls, re-reading context they already acted on.
Per-token got cheaper. Cost per finished task quietly went up.
Track the task. Starve the scar tissue.
Most agent bugs I debug aren't model bugs. They're scar tissue: an instruction from 3 weeks ago still in context, quietly overriding today's intent.
Habit most people miss: give every memory an expiry date. If you can't say why it's still there, delete it.
wrote up how anyray sits in the path — self-hosted middleware between your users/agents and the llm apis.
just-in-time context cleanup, compression, token attribution. no model swap required.
anyray.ai/blog/how-anyray-wo…
Computers should be used by agents, not humans. If you are still clicking buttons all around, you are just like the grandma that has hard time leaving her Nokia for iPhone
I was at the gym when my phone buzzed: a production alarm.
A few seconds later it buzzed again, a pull request with the fix. I hadn't
touched anything. No one had.
An AI agent on our on-call rotation caught the incident, investigated it, and
opened the fix on its own. The part that makes that safe to run: it can open
the PR, but it can't merge it.
The build:
CloudWatch alarm → SNS → Lambda → headless Claude Code on Bedrock. It reads the logs and recent git history with read-only creds, opens a PR on an incident/* branch, and asks in Slack: "here's what broke, here's the fix - approve?"
Never merges. Never deploys. Never touches main. Built in 15 minutes.