I make AI agents do real work, test what actually works, and show what it costs. Building tools along the way. Claude Code, Codex & more.
New York, USA
Joined March 2011
- Tweets1K
- Following92
- Followers98
- Likes622
Pinned Tweet
not wrong at all, i ran the same kind of thing with a 3d car configurator. opus 5.5 looked best but hit my 40 min cap at $5.76, haiku 5.5 got a working one for $0.38. did you track cost per model? curious how sol 6.1 compared on yours
opus 5.5 built this ship for $6.94. gpt-6.1 sol changed the sails for $0.20
one edit: "make the sails triangular, change nothing else"
sol did it in 2 minutes. hull, ocean, light and camera untouched
building is the expensive part. changing it doesn't have to be
tested exactly this. same prompt, both on high, 3 runs each: opus 5.5 $6.94, sol $0.53
opus does look better. the light, the haze, the detail. but sol gets you most of the way for 13x less, and opus hit my 40 min cap in 2 of 3 runs
better, yes. 13x better, not on this one
Note that 5.5 on high is a lot better than GPT 6.1 Sol on high, so though the cost gap is a lot smaller when you consider subsidization, you're getting a better model.
If you compare a lower reasoning effort on Opus to a higher one on Sol to be more fair, the Claude advantage widens.
Surprisingly competitive, tho
gpt-6.1 sol built this galleon for 53 cents. opus 5.5 cost $6.94
same prompt, 4 models, all on high effort, 3 runs each:
terra: 1m 25s, $0.07
sol: 16 min, $0.53
sonnet 5.5: 40 min*, $4.97
opus 5.5: 37 min, $6.94
*sonnet hit my 40 min cap in all 3 runs, opus in 2 of 3
opus has the best light. sol is 13x cheaper and close enough that it's the one i'd pick
Sergey | AI Builds & Tests retweeted
Psychedelic Neon Portrait
TFTT @EdithDel_Carmen
Here is my take..
Irregular rhythms.
Ether flowing between two stars.
Our pleasures, one infinity. ♾️
Inviting
@carolletta @RoseDenize @Stringerjohn007 @By0yom1
All invited 🪐
Psychedelic Neon Portrait
Prompt: A stylized neon portrait of a young adult woman centered in a close-up, three-quarter view, facing slightly left with her chin raised and eyes closed. She has blue-violet skin, full glossy lips, and straight dark hair framing her face. Oversized lime-green sunglasses sit beneath an enormous, broad-brimmed hat whose vivid pink and electric-purple concentric patterns dominate the upper frame. She wears a high, brightly striped neon choker and a vivid magenta-and-lime patterned garment over her shoulders. Several dark-furred chimpanzees with expressive faces surround her in the background, peering from the shadows at different heights. The composition is tightly cropped, with the woman in sharp focus and the animals partially obscured around her; the perspective is intimate and slightly low. Saturated ultraviolet blues, hot pinks, and fluorescent greens glow against a nearly black, jungle-like setting.
Made with @AdobeFirefly Animated with Gemini Omni Flash 1.1 Inviting: @NecoHashimoto @JimAlbany @Bic_Revelation Everyone is Welcome!
6 cents got me a toy. $3.78 got me a movie.
Pawel found Terra drops off hard below max effort. I hit the same wall at default.
One prompt, no help, a WWII plane chase in Three.js:
Terra (low): 56s, $0.06
Sol (medium): 17 min, $0.61
Sonnet 5.5: 39 min, $3.78
Opus 5.5: 40 min*, $5.11
*hit the 40 min cap in all 3 runs
I tested GPT-6.1 Terra on real work. 2 repos. 105 replanted bugs. Find and fix what you can.
The results:
- GPT-6.1 Terra (max): 40.7
- GPT-5.6 Terra (max): 32
- GPT-5.6 Terra (xhigh): 20
- GPT-5.6 Terra (high): 18
- GPT-6.1 Terra (xhigh): 11.3
- GPT-6.1 Terra (high): 9.7
It's cost and time effective. But right after max, its performance drops dramatically - far below Haiku 5.5 (xhigh).
you don't need opus for this. sonnet 5.5 built this jet engine in blender 12 times, all from code, about $0.57 a run
tried 3 add-ons people use to cut the bill too: ponytail $0.57, caveman $0.51, rtk $0.58 per engine. same quality, bills within ~10%
Everyone's using Claude Opus 5.5 to build entire worlds in #threejs.
I spent 7 days on a single human eye.
No textures. No 3D models. No photos. Just math, light and ~400 parameters.
Sound on 🔊 Link below 👇
measured what one 3d build actually costs. sonnet 5.5 wrote this jet engine in blender 12 times: plain, and 3 add-ons people install to cut their bill (ponytail, caveman, rtk). 3 runs each
median cost per engine:
plain $0.57
ponytail $0.57
caveman $0.51
rtk $0.58
1 run said ponytail cut the bill 30%. 3 runs said 0%. test an add-on once and you're mostly measuring luck
Stop burning Claude or GPT quota on basic 3D animations. Save this $0 pipeline:
1 - create a free model on Tripo or grab any from Sketchfab
2 - upload your mesh to Mesh2Motion, rig it, and apply existing animations
just a couple of minutes, literally $0 spent, and I have a ready-to-use snake for my project
measured how far it actually goes. same 8-file feature, default settings, share of one 5-hour window per task:
sonnet 5.5: 3% → ~33 tasks
haiku 5.5: 5% → ~20
codex sol 6.1 on chatgpt plus: 5.5% → ~18
opus 5.5: 15% → ~6
on pro the model you pick matters more than the plan
never thought i'd say this but claude's $20 plan is incredible value with opus 5.5
i've been on the 5x max plan for a long time but 2 weeks ago i realized i wasn't even hitting the 5-hour limit no matter how much i used opus 5.5, so i downgraded to the pro plan
it's much better value now than it was a few months ago
here's part of the answer. same 8-file feature on claude pro, default settings: opus ate 15% of the 5-hour window per task. sonnet ate 3%
claude pro, sonnet 5.5: ~33 tasks per window
claude pro, haiku 5.5: ~20
chatgpt plus, codex sol 6.1: ~18
claude pro, opus 5.5: ~6
haiku is the "cheap" model and still lost to sonnet
Anthropic should explain why Opus 5.5 usage limit is draining so fast and we got no reset.
Everyone tag @AnthropicAI @bcherny @claudeai @ClaudeDevs
tested 4 of the popular ones on real tasks. 3 didn't save money
sonnet 5.5, same bug + same 8-file feature, hidden tests, api prices:
ponytail: -34% on the big feature. but +20% on the small bug
caveman: cut output 21%. bill dropped 3%
superpowers: +32% on the bug. 1 more test passed
answer-me-with-html: never turned on in 6 runs
you mostly pay for the agent re-reading your code, not for what it writes
part 2 is running now. same savers, but the job is a 3D jet engine built from code
ive been messing around with ways to make Claude Code use fewer tokens (so I stop hitting my limit so early 😅)
one thing I noticed is how many tokens it wastes searching through files just to understand a repo
this weekend I hacked together a tool that gives it a codebase map instead. tested it on 6 repos and it actually decreased token usage quite a bit 🧵
tools tested:
ponytail v5.3.2 github.com/DietrichGebert/po…
caveman v4.0.1 github.com/JuliusBrussee/cav…
superpowers v7.0.0 github.com/obra/superpowers
answer-me-with-html v0.5.5 github.com/QingYunA/answer-m…
3 runs per tool per task, 6 for the bug baseline + caveman. small sample, so caveman's -3% is noise. on the big feature ponytail saved the most but never hit a perfect score, plain did once
Claude Code's creator shared these prompts. One of them actually made my code worse. I tested 2 of
@bcherny
's prompts on real coding tasks. One caught bugs in 3/3 runs. The other improved 0/6 — and broke a passing test once. Small sample, but I didn't expect that result. Full comparison below 👇
Claude Code's creator shared these prompts. One of them actually made my code worse.
I tested 2 of @bcherny's prompts on real coding tasks.
One caught bugs in 3/3 runs.
The other improved 0/6 — and broke a passing test once.
Small sample, but I didn't expect that result.
Full comparison below 👇
12 of 12 runs told me "done". 3 actually passed my hidden tests.
same 8-file feature, 4 agents:
codex sol 6.1 medium: 2 of 3
codex sol 6.1 low: 1 of 3
opus 5.5: 0 of 3
sonnet 5.5: 0 of 3
most said "full test suite passes". their own tests did. mine didn't.
how do you verify before you merge?
🤖 Made with AI
Sergey | AI Builds & Tests retweeted
Posted the full numbers here, with the method: nitter.cf/Moriarty_tech/status/2…
Is Opus 5.5 nerfed, or is it just eating limits faster? I checked my logs instead of vibes.
Claude Code token logs vs my limit tracker (2 Max accounts, % read every 2 min):
- Oct 11–16: 8.7M tokens per 1% of the weekly limit
- Oct 18–21: 8.8M per 1%
Same price per token. What changed is me: ~17% of the week per day before, ~32% now.
Quality is a separate question, Nerf Bench covers that. Has anyone else checked theirs?
the codex review part is the real cheat code. i measured it.
same 8-file feature, 4 agents, 12 runs. all 12 told me "done". 3 actually passed my hidden tests. all 3 were codex.
codex sol 6.1 medium: 2 of 3
codex sol 6.1 low: 1 of 3
opus 5.5: 0 of 3
sonnet 5.5: 0 of 3
opus wrote "the full test suite passes". its own tests did. mine didn't.
which one reviews your PRs?
The biggest cheat code in agentic coding is having Claude Opus 5.5 open a PR, wait for Codex to automatically review it, then fix whatever Codex finds. Repeat this a few times before merging.
Opus might be the best coding agent, but GPT Sol 6.1 is the best code reviewer. It catches things Opus misses.