The Marionette Benchmark: Sonnet 5.5 vs. Fable 5.1 vs. Opus 5.5 vs. GPT-6 Sol
We have a new leader! A marionette only works if the bar above moves the limbs below, and juggling only works if the balls obey gravity and the hands take turns. Each model writes the whole animation without ever seeing what it made.
Claude Sonnet 5.5 xHigh | (13m 15s / $1.06)*
Somebody is actually holding the bar, a gloved hand reaching down from above. The puppet wears a bowler hat, the first hat in this whole test, with a pinstriped suit. His knees bounce with the rhythm of the throws, so the whole body is in the act, whereas Fable's legs never move. Dust drifts through the spotlight. Gravity is close to Earth and the balls cast shadows on the floor. The one miss is the same as Opus, the puppets body is driving the bar rather than the reverse.
*1st attempt was on Max setting. It thought for sixteen minutes and handed back an empty file ($1.28)
Claude Fable 5.1 Max | (24m 27s / $5.80)
Had three weeks at the top. Still the closest to real gravity, but nothing is holding its bar and the legs never move. The stage and background also less detailed than Sonnet.
Claude Opus 5.5 High | (3m 51s / $0.50)
From last week, run on high after max came back empty three times. Still the only one that draws the scene in 3D.
GPT-6 Sol Max | (7m 11s / $0.41)
From last week. Its bar really does tilt, but two of its balls still pass straight through each other.
Nobody has made the bar drive the puppet yet.
The Marionette Benchmark: Opus 5.5 vs. GPT-6 Sol vs. GPT-6 Luna
Same test as two weeks ago. A marionette only works if the bar above moves the limbs below. Juggling only works if the balls obey gravity and the hands take turns. Each model writes the whole animation without ever seeing what it made. Fable 5.1 stays in the top left because nothing has beaten it yet.
Claude Opus 5.5 High | (3m 51s / $0.50)*
*On max reasoning it thought for twenty minutes and handed back an empty file, at $2.56 a try. I tried three times and got the same thing every time. On high it finished in under four minutes, and it's the most impressive build here. It's the only one that draws the scene in 3D, so the balls pass in front of each other instead of through each other. Gravity is close to Earth and the feet stay on the floor. But it moves the bar to follow the hands, which is backwards. The hands are supposed to follow the bar.
GPT-6 Sol Max | (7m 11s / $0.41)
The bar tilts and the strings move with it, and the hands actually throw instead of sitting still. But both hands throw along the same arc, so two of the balls keep passing straight through each other. Front facing body, side facing head. It also titled its own work "A balancing act. Three balls. One deadline."
GPT-6 Luna Max | (6m 32s / $0.02)
Clean side view, and the balls never overlap. The bar never moves, so the strings just hang there. It also throws about a third higher going one way than coming back, which makes the whole pattern lean.
Claude Fable 5.1 Max | (24m 27s / $5.80)
From two weeks ago. Still has the best gravity and the best lighting of anything we've run.
The Marionette Benchmark: Opus 5.5 vs. GPT-6 Sol vs. GPT-6 Luna
Same test as two weeks ago. A marionette only works if the bar above moves the limbs below. Juggling only works if the balls obey gravity and the hands take turns. Each model writes the whole animation without ever seeing what it made. Fable 5.1 stays in the top left because nothing has beaten it yet.
Claude Opus 5.5 High | (3m 51s / $0.50)*
*On max reasoning it thought for twenty minutes and handed back an empty file, at $2.56 a try. I tried three times and got the same thing every time. On high it finished in under four minutes, and it's the most impressive build here. It's the only one that draws the scene in 3D, so the balls pass in front of each other instead of through each other. Gravity is close to Earth and the feet stay on the floor. But it moves the bar to follow the hands, which is backwards. The hands are supposed to follow the bar.
GPT-6 Sol Max | (7m 11s / $0.41)
The bar tilts and the strings move with it, and the hands actually throw instead of sitting still. But both hands throw along the same arc, so two of the balls keep passing straight through each other. Front facing body, side facing head. It also titled its own work "A balancing act. Three balls. One deadline."
GPT-6 Luna Max | (6m 32s / $0.02)
Clean side view, and the balls never overlap. The bar never moves, so the strings just hang there. It also throws about a third higher going one way than coming back, which makes the whole pattern lean.
Claude Fable 5.1 Max | (24m 27s / $5.80)
From two weeks ago. Still has the best gravity and the best lighting of anything we've run.
The Marionette Test: Astra vs Fable vs. Muse Spark vs. Gemini
A marionette only works if the bar above moves the limbs below. Juggling only works if the balls obey gravity and the hands take turns. The model has to hold both in its head and write out the coordinates without ever seeing what it made. Although not a frontier model, we included Gemini 3.8 Flash so it feels included and won't form a complex it takes into adulthood 🫂
Claude Fable 5.1 Max | (24m 27s / $5.80)
Best of the four. Correct gravity, best lighting/shadows, the only one that built a proper two bar rig. But it drew four strings, its legs never move, and the top bar isn't connected to anything and nothing is holding it up.
GPT-6 Astra Max | (17m 42s / $2.03)
Nice suit, plenty of strings. Its near knee bends backwards. Gravity wise it looks like he's in outer space, so the balls hang like it's juggling on the moon.
Muse Spark 1.3 Max | (4m 42s / $0.18)
Most complete rigging, seven strings running to head, hip, both knees and both hands. Open curtains and a city skyline (best background other than Fable IMO). Middling physics, rough suit, but at least it has real arms unlike Gemini.
Gemini 3.8 Flash Max/Extended Thinking | (1m 18s / In-App)
Despite being a Flash model, Gemini has the best juggling animation outside of Fable. The balls also have faint shadows on the ground, which GPT-6 Astra failed to do. The only one that worked out that a strict side view flattens a cascade into a single plane, and it corrected for it. Good gravity. But the arms are bare sticks inside a sleeveless suit and the background is empty.
Nobody made the bar drive the puppet.
Inspired by @vinn_ayy for resigning from Boiling People Alive Labs and surrendering all his equity. wow.
The Marionette Test: DeepSeek V4.1-Flash Max | (5m 00s / $0.04)
130x cheaper than Fable. Five live strings, a real cascade with the hands taking turns, and the only entrant that admits someone is up there: two strings run off the top of the frame instead of a bar hovering in the void. But the hands barely move, and it set the loop to half the pattern's true length, so a ball teleports across the chest every 1.5 seconds.
The Marionette Test: Astra vs Fable vs. Muse Spark vs. Gemini
A marionette only works if the bar above moves the limbs below. Juggling only works if the balls obey gravity and the hands take turns. The model has to hold both in its head and write out the coordinates without ever seeing what it made. Although not a frontier model, we included Gemini 3.8 Flash so it feels included and won't form a complex it takes into adulthood 🫂
Claude Fable 5.1 Max | (24m 27s / $5.80)
Best of the four. Correct gravity, best lighting/shadows, the only one that built a proper two bar rig. But it drew four strings, its legs never move, and the top bar isn't connected to anything and nothing is holding it up.
GPT-6 Astra Max | (17m 42s / $2.03)
Nice suit, plenty of strings. Its near knee bends backwards. Gravity wise it looks like he's in outer space, so the balls hang like it's juggling on the moon.
Muse Spark 1.3 Max | (4m 42s / $0.18)
Most complete rigging, seven strings running to head, hip, both knees and both hands. Open curtains and a city skyline (best background other than Fable IMO). Middling physics, rough suit, but at least it has real arms unlike Gemini.
Gemini 3.8 Flash Max/Extended Thinking | (1m 18s / In-App)
Despite being a Flash model, Gemini has the best juggling animation outside of Fable. The balls also have faint shadows on the ground, which GPT-6 Astra failed to do. The only one that worked out that a strict side view flattens a cascade into a single plane, and it corrected for it. Good gravity. But the arms are bare sticks inside a sleeveless suit and the background is empty.
Nobody made the bar drive the puppet.
The Marionette Test: Astra vs Fable vs. Muse Spark vs. Gemini
A marionette only works if the bar above moves the limbs below. Juggling only works if the balls obey gravity and the hands take turns. The model has to hold both in its head and write out the coordinates without ever seeing what it made. Although not a frontier model, we included Gemini 3.8 Flash so it feels included and won't form a complex it takes into adulthood 🫂
Claude Fable 5.1 Max | (24m 27s / $5.80)
Best of the four. Correct gravity, best lighting/shadows, the only one that built a proper two bar rig. But it drew four strings, its legs never move, and the top bar isn't connected to anything and nothing is holding it up.
GPT-6 Astra Max | (17m 42s / $2.03)
Nice suit, plenty of strings. Its near knee bends backwards. Gravity wise it looks like he's in outer space, so the balls hang like it's juggling on the moon.
Muse Spark 1.3 Max | (4m 42s / $0.18)
Most complete rigging, seven strings running to head, hip, both knees and both hands. Open curtains and a city skyline (best background other than Fable IMO). Middling physics, rough suit, but at least it has real arms unlike Gemini.
Gemini 3.8 Flash Max/Extended Thinking | (1m 18s / In-App)
Despite being a Flash model, Gemini has the best juggling animation outside of Fable. The balls also have faint shadows on the ground, which GPT-6 Astra failed to do. The only one that worked out that a strict side view flattens a cascade into a single plane, and it corrected for it. Good gravity. But the arms are bare sticks inside a sleeveless suit and the background is empty.
Nobody made the bar drive the puppet.
Left Image: A popular post of everyone dunking on @edzitron for 'incorrect predictions'
Right Image: Sam Altman's brother @jaltma saying the smartest people in AI constantly change their opinions because things move too fast
As an objective third party, there is no world where Ed Zitron is going to constantly talk all the time and still be right all the time. AI doesn't work that way.
