Legit (@legit_api) header
Legit (@legit_api) profile picture

covering AI news, working on @voxelbench

Joined October 2018
Photo by Legit (@legit_api)Photo by Legit (@legit_api)Photo by Legit (@legit_api)Photo by Legit (@legit_api)Photo by Legit (@legit_api)Photo by Legit (@legit_api)Photo by Legit (@legit_api)Photo by Legit (@legit_api)Photo by Legit (@legit_api)Photo by Legit (@legit_api)
can't get enough of the new models dw we're not exclusively testing SVGs but you have to admit it's pretty great!
Some new Claude models have recently entered early testing. A newer Fable checkpoint + new Opus h/t to kostikz on discord for these Fable outputs:
31
7
2
282
10,782
Some new Claude models have recently entered early testing. A newer Fable checkpoint + new Opus h/t to kostikz on discord for these Fable outputs:
71
25
43
755
164,533
Carbon is a nice improvement over Argon I did not like Argon at all, while Carbon has reduced slop, plus it's more careful now and adheres to constraints better but google should seriously consider adding xhigh and max reasoning levels make use of that 1M output token limit!
20
5
272
13,137
Introducing Gemini 4 Carbon, it's a big jump over Argon which we released just 2 weeks ago. Gemini 4 Carbon is SOTA on cyber, long horizon tasks, ranks #1 on Artificial Analysis V5, significantly improves on usability in domains like coding, and is more aligned.
1,223
1,156
857
15,861
1,231,339
saw some wild shit today
11
1
1
78
5,277
Anthropic has filed an S-1 registration statement with the SEC related to its proposed IPO. Read more: anthropic.com/news/s1-sec Legal disclaimer: A registration statement relating to these securities has been filed with the SEC but has not yet become effective. These securities may not be sold, nor may offers to buy be accepted, prior to the time the registration statement becomes effective. This post does not constitute an offer to sell or the solicitation of an offer to buy these securities, nor shall there be any sale of these securities in any state or jurisdiction in which such offer, solicitation or sale would be unlawful prior to registration or qualification under the securities laws of any such state or jurisdiction.
1,344
2,374
2,431
18,267
9,477,318
Models can seemingly degrade over time. The following scenes were created with the same prompt and the same model, @GoogleAI's Gemini 3.8 Flash, by a user on DesignArena. The first was generated just 3 days after release, while the second came almost a month later. The difference is striking. The first scene features a finished bungalow with terraced gardens, while the second is barely recognizable. Digging into the code, we find that the second scene still attempts to build the garden, windows, and patio, but buries much of it beneath its own terraces. In aggregate, the share of objects never visible on camera jumps from 18% to 57%. For bushes, the share placed underground jumps from 0% to 100%, and even the house walls go from fully above ground to 64% below it. Both scenes share the same simple user request, yet a few weeks after release, the model struggles to fulfill it. Try making your own generations and judge for yourself at DesignArena.ai!
39
32
16
396
27,881
Kimi K3.1 Pro has ranked 6th on the VoxelBench Live Leaderboard! currently tied with Gemini 4 Argon
4
2
3
72
8,583
Kimi K3.1 Pro is now on VoxelBench Happy voting!
3
1
53
1,728
🚨🚨ALERT: ANTHROPIC IS A/B TESTING USAGE LIMITS Many users on X are reporting LOWER usage limits right now. THEY ARE NOT WRONG. We detected an A/B test going on that DECREASES your monthly API-value by ~20% while INCREASING your 5h usage limits by ~20%. This only affects SOME accounts. (1/2)🧵
202
198
161
3,235
515,454
tfw the missing piece is still missing
3
38
3,454
Took some screenshots for you all VoxelBench 2 is now in the final stages of development, beta test coming soon!
Fable 5.5 is a monster on VoxelBench here's how it did on 1 of our prompts:
10
5
3
126
6,323
Fable 5.5 is a monster on VoxelBench here's how it did on 1 of our prompts:
58
40
18
743
99,538
here's Opus 5.5 for comparison.
3
1
95
5,832
we've been cooking with Fable 5.5 got some crazy stuff to show tomorrow
15
11
322
17,719
Fable 5.5, the missing piece
11
1
1
172
6,140
Google doesn't let gemini think for more than single digit minutes Anthropic makes sure claude overthinks OpenAI makes sure their model ships the sloppiest code on earth that WILL break after a week or 2
11
3
141
6,392
Argon is out on VoxelBench its performance is very uneven; sometimes producing clean and detailed builds, while also often making much less detailed and smaller builds overall im underwhelmed, but I hope they recognize this gap and improve it fast
Gemini 4 Argon is now live on VoxelBench!
5
54
3,465
one similarity I found interesting (and annoying) between this model and Fable 5.0 is that they both prefer to make smaller builds but unlike Fable, Argon made no effort to verify that the blocks it placed weren't overlapping or clipping into other blocks on average, Argon spent 2 minutes per VoxelBench prompt; far below the average of models like Opus and Fable, and even Astra or Sol
7
1,181
Gemini
1,050
387
517
6,634
1,266,892