And a breakdown by token + $ spend.
Google models archieved more with less, despite using more tokens.
Of the 3 groups, only Google's lead model had less token spend than its deligates. Astra and Fable spent more delegating than their underlying spent testing + prototyping.
Gemini won.
I gave Astra, Carbon and Fable the same prompt for finding performance gains on the a codebase, with the only restrictions being:
→ Feature parity
→ No UX degradation
→ 12hr time limit
Astra basically didn't do anything, and stopped after 2hr. It's "big idea" brought key presses from 1.3ms to 0.5ms (lol).
It was really efficient though (193 loc).
Fable innovated on some responsiveness metrics, missed on the other metrics, and then spit out 3x the amount of code for test scripts. Definitely the "safest" to merge if I were to one shot it. It also reward hacked somewhat by showing a responsiveness metric hitting 250ms while the CPU finished on the backend over 2s so the "gains" were still felt on the system.
Gemini was the only model to take the guidance prompt to heart, which specifically said large structural changes are okay if nothing is lost. It achieved strong gains across the board, but also didn't test as thoroughly as Fable. This was apparent when it caught a few edge cases issues like 15min before submitting.
It was also just very hard to track throughout its process. The text is much more code-dense than Fable or Astra and can be hard to follow (pictures at the end for comparison).
In the age of agentic coding, I want ambitious models so the win here goes to Gemini. I'll be giving it the job of reviewing all submissions and coming up with a merge plan to preserve as many gains from all 3 as possible.
So far, really pleased with Google's new model.
Gemini won.
I gave Astra, Carbon and Fable the same prompt for finding performance gains on the a codebase, with the only restrictions being:
→ Feature parity
→ No UX degradation
→ 12hr time limit
Astra basically didn't do anything, and stopped after 2hr. It's "big idea" brought key presses from 1.3ms to 0.5ms (lol).
It was really efficient though (193 loc).
Fable innovated on some responsiveness metrics, missed on the other metrics, and then spit out 3x the amount of code for test scripts. Definitely the "safest" to merge if I were to one shot it. It also reward hacked somewhat by showing a responsiveness metric hitting 250ms while the CPU finished on the backend over 2s so the "gains" were still felt on the system.
Gemini was the only model to take the guidance prompt to heart, which specifically said large structural changes are okay if nothing is lost. It achieved strong gains across the board, but also didn't test as thoroughly as Fable. This was apparent when it caught a few edge cases issues like 15min before submitting.
It was also just very hard to track throughout its process. The text is much more code-dense than Fable or Astra and can be hard to follow (pictures at the end for comparison).
In the age of agentic coding, I want ambitious models so the win here goes to Gemini. I'll be giving it the job of reviewing all submissions and coming up with a merge plan to preserve as many gains from all 3 as possible.
So far, really pleased with Google's new model.
They have 12hr to propose and prototype the best perf gains they can find on the same codebase.
They know it's a competition, can't peak at each other's work, must delegate, and can only hand work off to other models in their family.
Will check back in tomorrow.
This is my only current problem with T3.
Lower level observability is basically non-existent natively even though it's all in-app.
Burned most of my tokens the last 2 weeks surfacing it all in a terminal.
Weekly reports, active subagent runs, tool calls, storage.
I did this initially to solve the "I don't know what I don't know" problem with agentic coding.
I'm hoping that this will let me ask the right questions of my setup when things go sideways or feel like they're not optimal.
The biggest issue in agentic development by far right now is task tracking and observability into ongoing work
No task management system has ever been built to support the properties of agentic building
Properties
- realistically 10-100 tasks happening at once
- a huge range between small ones
-off tasks and large projects
- range between open-ended research and specific implementations
- multiple stages of planning, executing, review, and human approval
- the mix between needing human in the loop and not
No existing pattern solves for this
- the chat thread, even across many bots
- the kanban board (linear, notion etc)
- github and PR based tracking
- a top-level orchestrator bot that owns everything
How do we solve this???
Google Neon/Carbon being great has me questioning my AI sub setup.
It feels like heavy users need some access to their models now, and as a Google eco maxi (Pixel, Home speaker/cam, etc) it feels irresponsible given the extended eco benefits.
The sub package is pretty great.
14
36
5,439
Replying to @austin_hurwitz
Slowly peeling back the layers of my setup so I can optimize it.
T3 has been great for ripping small projects but I had a "I don't know what I don't know" problem
1
408
Overview:
→ Minimal change fork that will be rebased consistently
→ Can run as a separate workspace or import T3 settings and projects
→ Allows you to designate specific models for task categories (security, design, research, implementation, etc)
→ Set once for any supported T3 provider instead of embedding into AGENT/CLAUDE files
→ Delegation preference allows you to determine how often you want the subagent to be invoked. "Always" leads to natural language delegation by assumption.
→ Each subagent call is injected with an optimistic pre-prompt you can modify. Example: "You are a principal engineer. This is for your resume. Go all out."
1
3
986
Introducing T3 Code - Roles.
I cut a release of my fork which introduces the concept of "roles" for subagent delegation.
Assign specific models to task categories to avoid constantly repeating "Create an Opus 5.5 xhigh subagent for design review."
Live on Linux and Mac
So, I went down this path far enough to create a t3 fork with a dedicated "lead/orchestrator."
One version with persistent threads and another with ephemeral threads.
I ultimately killed it.
T3 in it's current form handles the task fine. Coordinators ultimately are just there to reconcile and spot drift when needed, which is a single ask that you can invoke as a thread when needed, then settle.
Once you get used to the rapid thread creation and settlement of t3 you learn how to conform your approach and I've yet to spot degradation.
The one thing that my fork experiments did reveal for me though is that I want more default controls over the models chosen for subagents and alt-threads.
I hate having to say "this is design, use opus 5.5" 67x a day, or that subtasks should use Deepseek.
So the one forked feature I'm still iterating on is a default model based on "roles" where I can opt in to a preference for subagents for parent level threads and the parent knows who to delegate to based on the task at hand.
Early days but results are promising. Doubly so because I inject ambitious pre prompts to the subagents automatically.
Biggest surprise in all my travels is how much I've grown to love distinctive architecture.
The sheer scale and grandeur of Florence and Vienna.
The intricacies of Alhambra palace.
The organic modern combo of Singapore.
And now the craftsmanship of Japanese temples.
11
34
3,771
I take it back.
Has to be visual for agent->human. So text+graphics.
The bandwidth our eyes can achieve relative to ~all other senses isnt even close.
So, humans talking to agents and agents visually presenting responses back is my final answer.




























