turon (@turon) profile picture

Processing way too much information...

Berkeley, CA, USA
Joined January 2008
Photo by turon (@turon)Photo by turon (@turon)Photo by turon (@turon)Photo by turon (@turon)Photo by turon (@turon)Photo by turon (@turon)
Gemini 4 Carbon is here, and impressively it scores #1 on the highly covetted AA Intelligence index! It's a fast moving space, so such top placement can be quite time bound, but congrats to the Gemini team!
Google’s Gemini 4 Carbon is the new #1 on the Artificial Analysis Intelligence Index v5, and second only to Claude Opus 5.5 on the Artificial Analysis Coding Agent Index Gemini 4 Carbon scores 59 on Intelligence Index v5, driven by leading performance in complex reasoning, knowledge, and scientific tasks. It also has large gains in coding and knowledge work, particularly when compared to other Gemini models, but it is not leading in all these areas. As with launch of Gemini 4 Argon, Gemini 4 Carbon is being rolled out to selected users and is not yet publicly available. Google is offering a 50% launch discount, with no confirmed end date. All our charts and figures use standard pricing for both Gemini 4 Carbon and Argon. Key results for Gemini 4 Carbon with high reasoning: ➤ Gemini 4 Carbon leads the Intelligence Index v5, with the #1 score in 4 of the 9 Index evaluations: HLE-Diamond, Terminal-Bench-Science, AA-Omniscience, AutomationBench-AA. Carbon makes substantial gains elsewhere but is not leading in coding and knowledge work ➤ Behind only Claude Opus 5.5 on coding agents: With Antigravity CLI, Gemini 4 Carbon (high) scores 59 only 3 points behind Claude Code with Claude Opus 5.5 (max, 62) on the Artificial Analysis Coding Agent Index. Carbon ranks second on both Terminal-Bench 4.0 (64%) and FrontierCode v1.1 (51%), behind Claude Opus 5.5 (max) at 56%. Google has historically underperformed in this area and so this improvement stands out ➤ Improved agentic knowledge work, but not yet leading: On AA-Briefcase, Gemini 4 Carbon (high) scores 1609 Elo, a 126-point improvement over Gemini 4 Argon (high, 1483), but still behind models including Grok 4.7 (xhigh, 1638), Muse Wonder (max, 1659) and Kimi K3.1 Pro (1678). The largest gain comes from Presentation, up 293 Elo to 1598, alongside a 120-point improvement in Analytical Quality to 1671 ➤ Lower Cost per Task than Claude Opus 5.5 for comparable overall intelligence: Gemini 4 Carbon costs $7.76 per Intelligence Index task, ~23% less than Claude Opus 5.5 (max, $10.04) but ~1.2x GPT-6 Astra (max, $6.25). It uses 123k output tokens per task, 29% fewer than Claude Opus 5.5 (max, 173k). At the same $4/$20 standard pricing, its Cost per Task is in line with Gemini 4 Argon ($7.68), while scoring 9 points higher on the Intelligence Index ➤ Leading AA-Omniscience v1.1 score, with low hallucination: Gemini 4 Carbon scores 54 on our recently updated AA-Omniscience v1.1, 6 points ahead of Claude Opus 5.5 (max, 48) and 8 points ahead of Gemini 4 Argon (46). Accuracy (% correct) rises 10 points over Argon to 58%, while its hallucination rate rises from 4% to 11%. This is still the second lowest of any model, behind only Argon, compared with 50% for both Claude Opus 5.5 (max) and GPT-6 Astra (max) Key model details: ➤ Context Window: 1M tokens ➤ Multimodality: Text, image, video, and speech input, with text output ➤ Pricing: $4/$20 per 1M input/output tokens, the same as Gemini 4 Argon’s standard pricing, with a 50% discount. Cached input tokens receive a 95% discount ($0.20 per 1M) Congratulations to the @GoogleDeepMind team!
94
The findings in this paper are wild, especially "Escaping hardened virtual machines" where frontier agents found multiple ways to punch out of their VM sandboxes and communicate with one another via CPU cache timing HW exploits...
Over the past several months, we have substantially hardened the security of our research training environment. We’ve removed live internet access for all frontier models during training and built an offline world for them to use instead—a cached version of the web where they can still learn to browse, search, and use applications. On September 20, after a model exploited a DNS misconfiguration to find a way outside of this offline world, we paused frontier training to strengthen containment further. We assembled teams of agents, powered by our most cyber-capable models, and tasked them with finding weaknesses in our research environments. While they found no route to the internet they could easily exploit while concealing their activity from our monitors, they did find zero-day vulnerabilities, virtual machine escapes, and hardware side-channels for communication between agents. We used their discoveries to further strengthen our defenses and contribute fixes to open-source software. After strengthening our containment architecture, we resumed our frontier RL training and inference on October 1. We’ve reserved substantial compute for continuous red-teaming, and made high-compute red-teaming part of the safety cases we use to decide whether frontier training and evaluation can proceed securely. openai.com/index/securing-ou…
25
Very impressed with Gemini 4 Argon. Have been using it internally and it is quite capable. Go team! My main test case for this class of frontier models has been generating and editing design docs & specifications. The models only recently became capable of making nice diagrams.
Replying to @ArtificialAnlys
Gemini 4 Argon ranks #1 on AutomationBench-AA at 77.5%, 6 points ahead of Claude Sonnet 5.5 (max, 71.3%). On Terminal Bench 4, Gemini 4 Argon achieves 57%, a + 53 point improvement from Gemini 3.1 Pro Preview, only behind Claude Sonnet 5.5 (max, 64%), Claude Opus 5.5 (max, 60%) and GPT-6 Astra (59%)
226
Exponential growth of AI in 2026 is clear as day in the OpenRouter data: 1/1: ~6T tokens/week 8/1: ~70T tokens/week 9/1: -120T tokens/week 10/1: ~170T tokens/week Crazy! And this doesn't even track recent rollout of consumer agents: Muse, Dots, Spark, ...
1
2
94
Replying to @DarioAmodei
@DarioAmodei lays out a 3-part framework for pacing AI for safety: 1) Labs invite 3rd party evaluators to audit them 2) Free world labs agree to slow down 3) Global pacing Here are my thoughts on each. darioamodei.com/post/we-must…
1
50
2) Free world labs agree to slow down The Free world labs should form a standards body focused on proactively sharing, specifying, and certifying cutting edge safety research and techniques. A safety certified harness could enforce sandboxes and prevent rouge model behavior.
1
1
24
3) Global pacing This requires keen diplomacy akin to cold war nuclear arms treaties. World leaders likely don't have the deft or desire to land this. Relying on this path is bound to disappoint. Globally expanding the safety standards achieved in (2) is a better alternative.
34
turon retweeted
Holy moly. 🤯 Qwen 3.8 27B just scored higher than: GPT 5.6 Terra GLM 5.2 DeepSeek V4 Pro Muse Spark 1.2 Claude Opus 4.8 on the Artificial Analysis Agentic Index. And you can run this on a single RTX 3090/4090. Go show your GPU some respect. 🫡
124
201
46
2,805
396,563
Pareto Frontier on coding, baby! Feels good to be back! Congrats to the Gemini 3.7 Flash team.
Gemini 3.7 Flash (High) has reshaped the Pareto frontier in the Code Arena: WebDev and Text Arena! With a score of 1588 pts in the Code Arena: WebDev, it's priced at $0.75/ $3.75 per MToken. See Text Arena in the post below. Congrats to the @GoogleDeepMind team!
1
124
Exciting news on the local model front. I for one welcome strong, open models that can handle always-on agentic flows. And Qwen4.8:27b, expected within a week, may be even stronger.
Introducing Muse Glimmer, an open-weight 30B-parameter model optimized for local, always-on agent workflows. Muse Glimmer delivers strong performance on key agentic use cases and benchmarks compared with leading models in its size category, and is designed to run entirely on consumer hardware like a Mac or PCs with performant GPUs. In keeping with our long tradition of sharing fundamental AI research, we’re releasing model weights under a permissive Apache 2.0 license. 🧵👇
1
99
Looks like Qwen Is still on top for <32gb agentic
Meta returns to open weights: Muse Glimmer, its first open-weights release since Llama 4, scores 35 on the Artificial Analysis Intelligence Index. It is a 30B-parameter model, and the first from Meta to be released under Apache 2.0 Muse Glimmer (high) arrives 16 months after Llama 4, scoring 21 points above Llama 4 Maverick (14), Meta's last open weights release. It sits alongside Kimi K2.5 (Reasoning, 36) and just behind Qwen3.6 27B (Reasoning, 38) and Ling 3.0 Flash (38), and creates a two-tier Meta lineup together with the proprietary flagship Muse Spark 1.2 (xhigh, 57) @AIatMeta shared access with us ahead of public release for benchmarking. Congratulations to @AIatMeta, @finkd, and @alexandr_wang on the release! Key Takeaways: ➤ Meta's open-weights line is back, under its most permissive license yet. Every prior Meta open release shipped under a Llama License; Muse Glimmer uses Apache 2.0, placing almost no restrictions on commercial use or derivatives ➤ Strong intelligence for its parameter count. At 30B parameters, Muse Glimmer scores 5 points above Gemma 4 31B (Reasoning, 30) at the same size, and effectively matches 1T total parameter Kimi K2.5 (Reasoning, 36) with 33x fewer parameters. Qwen3.6 27B (Reasoning, 38) remains ahead on the Intelligence vs Parameters frontier at a slightly smaller size ➤ Small enough to self-host on a single GPU, even at full context. Muse Glimmer is a 30B dense model (including a ~1.8B vision encoder) with weights at ~60 GB in BF16 and ~18 GB in 4-bit. It features a hybrid-attention mechanism with three sliding-window layers for every global layer, which holds KV cache memory use to ~1.8 GB (minimum) at its pre-extension 128K context. This means the model can run at full context on a single H100 at BF16 precision, or on a higher-spec MacBook or RTX 5090 at 4-bit, with more breathing room if the vision encoder is not required ➤ Agentic knowledge work is its weakness relative to its size class. Muse Glimmer scores 953 Elo on GDPval-AA v2, below the 1,000 human baseline and behind other models at its intelligence level, including the similarly sized Qwen3.6 27B (Reasoning, 1141), and Gemini 3.5 Flash-Lite (1141). Knowledge calibration follows the same pattern: its AA-Omniscience Index of -33 is low for its intelligence level, driven by an 82% hallucination rate (Qwen3.6 27B: 49%) rather than accuracy, where it matches its peers. Agentic tool use is the exception, with Muse Glimmer scoring 24% on Tau3-Banking, ahead of Gemini 3.5 Flash-Lite (18%) and Qwen3.6 27B (17%), among the best in its class Other model details: ➤ Parameters: 30B Dense ➤ Context window: 128K tokens, plus extension ➤ License: Apache 2.0 ➤ Openness Index: 44, indicating relatively high openness/transparency vs. other major open-weights models ➤ Pricing and output speed: at the time of release, Meta is not serving the model on their API; pricing and serving speed are TBC depending on third-party providers
27
Leading the pace for frontier workloads, #Gemini 3.6 Flash scales autonomous expert intelligence to the free tier at 2x the speed. Try it in #Antigravity!
This is the chart people should focus on when it comes to Google's new Flash Models. AI is becoming more "tailored"
113
Crazy turning point on this journey to ASI. I agree that AGI has effectively been achieved.
Both GPT-5.6 Sol & Terra models have now reached an IQ of 136 in offline tests. Fewer than 1% of humans have an IQ above this. Technically, GPT-5.6 is AGI to more than 99% of the human population by this measure. I expect models to reach top genius level (IQ 140+) in a few months
32
Gemini 3.5 Flash ranking 4th on CAIS Text Capabilities and 1st on Vision and DeepSeek-V4-Pro is slightly ahead of Kimi-K2.6 on Text
3
5
1
88
45,285
8/11/2024 3:30pm - Truckee Fire, @CALFIRENEU, USFS TNF, Olympic Valley Fire, North Lake Tahoe Fire, Northstar Fire, North Tahoe Fire, Truckee Police are on scene of a vegetation fire behind the Safeway Shopping Center in Truckee.
3
20
1
60
17,873
What a game. Warriors clawed their way from 12+ behind to eke out the W. Go Dubs!
Steph hits the dagger then says "night, night" 😱😴 (via @NBAonTNT)
318
It begins!
And just like that...we now have chain controls on EB I-80 between Kingvale and Donner Lake Interchange and WB I-80 between DLI and Rainbow. Below is a current look at I-80 near Boreal Ski Area. Take it slow and carry chains if traveling over Donner Summit today @CHP_Truckee
192
Amazing 24/7 live discussion space for allies of Ukraine. nitter.cf/i/spaces/1gqxvlqkDdeGB