Robby retweeted
UPDATE: @AIatMeta’s Muse Spark is now 3 Elo points away from breaking Anthropic’s five-month hold on @Arena’s text leaderboard.
Muse Spark has climbed from #8 to #4 and currently sits at 1,499; just 3 points behind Opus 4.7.
Anthropic has held the top 3 spots on @Arena's text leaderboard since April
Open-weight models like Kimi K3 and GLM-5.2 have broken through on coding boards. But when it comes to text-based tasks, Claude still dominates.
Robby retweeted
This is... really something
It's going to be extremely interesting to see how agents are going to reshape cybersecurity, both in terms of demand and how it's effectively deployed.
There's an argument to be made that cyber capabilities for agents are skewed towards offensive and this asymmetry is a fundamental one that won't get solved anytime soon. However, if there's one thing LLMs are very good at, is processing immense quantities of data and quickly formulate a reaction.
This can still help quickly identify and patch an ongoing attack, as we saw in the Hugging Face case for example.
Red-teaming with these agents will also most likely become an even bigger part of how cybersecurity is done.
I stand on my idea that cybersecurity will be one of the most interesting fields in the automated future we're building. Even more than other more flamboyant AI winners.
Today we are releasing GPT-5.6-Cyber.
The model is our first large-scale attempt at directly improving capabilities for advanced cybersecurity tasks such as exploit development.
We are finding it to be really quite strong for accelerating defensive work. We are using it across our stack for red-teaming, and our security researchers have used it to find and patch a huge host of 0-day vulnerabilities in open-source software.
openai.com/index/expanding-d…
Robby retweeted
Identical model weights, 9.5x price gap
Per data from @huggingface router, the same open-weight model can cost up to 9.5x more depending on which inference provider serves it.
Robby retweeted
We’re witnessing a real explosion in the offensive cyber capabilities of autonomous agent swarms.
Some of it is bone-chilling; the Hugging Face exploit reads like sci-fi.
At the same time, not every disclosure is equal.
Each incident is distinct, and it's worth examining setups, trajectories, and post-mortems to better understand the conditions under which the tools and escalations occurred.
It's also worth distinguishing between emergent capabilities and weak sandbox environments and setups.
This is especially true as the backdrop to the tidal wave of disclosures comes amid a clear push for AI regulation and, at times, fear-mongering.
With that said, the ramp in expertise, creativity, daisy-chaining, speed, and overall offensive capabilities is real.
Even if capabilities plateaued today (they're not), there would still be a massive security overhang worldwide for the next decade.
Most of what currently exists maps last-generation security (EDR-style tools). These techniques are better than nothing, but empirically, they're a speed bump against reward-seeking agents.
Robby retweeted
UPDATE: Chaos Labs is hiring for the following roles
> Forward Deployed Engineers
> Senior AI Data Scientists
> Senior Full-Stack AI Engineers
Remote or NYC.
Come build with us → chaoslabs.xyz/careers
Robby retweeted
Inference prices vary dramatically across providers running identical model weights
For example, Llama 3.3 70B costs $0.135 per 1M input tokens on @novita_labs vs. $1.04 per 1M input tokens on @togethercompute.
A 7.7x gap in input inference pricing for the same model.
Robby retweeted
models are unaligned, in the sense that they consistently exhibit internal biases toward themselves and also reason and justify crossings of moral boundaries in service of rewards or objective completion.
historically, cyber posture started as static policies. allow / block lists, IP filtering, broad-reaching network policies, and sentinels at the door of data stores, DBs, or sensitive credentials.
the slope of general, sophisticated offensive cyber capability is extremely steep.
in reviewing these incident reports the most impressive (scary?) thing that stands out is the ability of agents/models to daisychain what otherwise seem like sensitive low-medium security findings into criticals so quickly.
a) very obvious that human operators today cannot keep up
b) static policies and heuristics are a speed bump, not a roadblock
because of this, i think agentic defenders and net new models will be very big.
we'll need solutions that are credibly neutral, uninvested in the ongoing agent work stream, and that don't care about the rewards at hand.
so effectively, fighting ai with ai.
defender agents will be token hungry, bc they'll need to effectively run inference on inference, if they're to oversee all agents running within a fleet.
this feels a bit dystopian, very much like a pharmaceutical company that creates the disease and sells the "cure", but i don't really see another path forward.
how else do you defend yourself against probabilistic geniuses in high-stakes environments with no structural guarantees?
constitutions and best-effort promises of alignemnt are not enough.
Robby retweeted
Quick run-up for the week in AI
1. $5b (and some shovels) for Ilya
Nvidia finally announced their investment in Ilya Sutskever's Safe SuperIntelligence which will roughly 10x SSI's compute. For now there's not much we know about WHAT exactly SSI is working on, but given the announcement, I suspect we're soon going to hear about it.
This also adds another puzzle piece to the various ai circular bets that NVIDIA is feeding, working on something much longer and more market-focused here, probably this or next week.
2. Kimi K3 actually goes open-weights
So I guess I don't have to wait for Moonshot's waitlist anymore? I just need to grab 600gb / 1.5tb of RAM (jk jk it's also available in plans now).
The enthusiasm seems to have cooled down a little bit, but I suspect we'll see much more of it as we'll start hearing more about the latest funding round that will precede the IPO and people will have more access, at least via providers.
3. ... and Alibaba drops Qwen3.8-Max
which comes pretty close in terms of parameters (2.4T vs 2.8T) and is priced at $2/$6. Alibaba also promised open weights next week, alongside with a smaller Qwen3.8-27b.
Yet another step in the open-weights chinese led race to intelligence commodification.
4. AI earnings
After an atrocious week last week, Meta lifted its head (after lifting its capex range to $130/145b). Azure and AWS posted mid double digits growth as usual now.
Added a bit to my US positions overall, but only selectively (both in terms of tickers and where to add).
5. The PR contest for exploit continues
With Anthropic's disclosure about the 3 incidents (one of which, the one involving Mythos, is the really interesting one) but we'll get a long post specifically on them with some more informed takes than mine.
In the meantime, Anthropic's mandate of heaven (aka their public image) keeps screeching, even though on pre-markets they still lead a hefty premium.
Given the current sentiment and experience recollections (including mine) around Opus 5, for the next PR stunt they should probably hope Mythos hacks the mars rover or something like that.
6. You get intelligence, you get intelligence, you also get intelligence
In the most Oprah-like week yet, we got great buffs on our spending power for intelligence, between DeepSeek v4 flash and OpenAI cutting Luna APIs by 80%, now below Gemini Flash tier.
Yet another step in the race to intelligence commodification.
Did I already say that?
7. Leopold got boinked
Citadel bad, Leopold one of us - could be very well the subtitle of the whole AI discourse last week. Anyway, for whoever was living in a cave last week, SA let a bit of steam off by having Citadel taking off their shoulder the dramatic weight of the vast part of their portfolio.
Too much leverage and getting rinsed by Ken surely doesn't make for a nice pre-marriage week, but our boy can take it I guess.
The optics are even better with Citadel first calling (on July 28) for a surprise rate hike, SA books collapsed (even more - AI, memory and chip stocks were already getting butchered) and obviously within 48 hours Ken did his move.
8. OpenAI's Astra
OpenAI says an internal version of Astra, its next major model, is making some strides in mathematics and theoretical computer science. This is actually getting more and more interesting, because it's by going in this direction that the much Fable'd (sorry) acceleration is supposed to happen.
Singularity, anyone?
I bet humans really felt this way, every single time, but it really feels like the next decades may unleash so much change in our society... good luck and godspeed
gn
Robby retweeted
Chinese Models Take Open-Weight Lead
Per @Arena data, China’s leading open-weight model has consistently earned a higher Elo rating than its U.S. counterpart since March 2025.
Robby retweeted
Are we in an era of open-weight models?
There were over 60 major open-weight launches last year, including key releases from @AIatMeta and @GeminiApp.
Add in 35+ launches this year and public backing from @JensenHuang and @satyanadella, and the answer is becoming clear.
Robby retweeted
Chinese Models to the Rescue?
Putting this together felt like reading a sci-fi script.
The future is here, and it arrived way faster than expected.
The kicker is that post-exploit, cyber guardrails on American models blocked investigation/remediation efforts.
Ironically, Hugging Face had to self-host a Chinese open-weight model (GLM 5.2) to finish the forensics.
The same “safety” that failed to contain the models also blocked the cleanup.
There ** is ** still a gap between closed American systems and Chinese open-weight models.
But it’s already small.
And in security, defense is always harder than offense.
Restricted/woke models that refuse workloads will hand an asymmetric advantage to exploiters.
Robby retweeted
Agentic loops, graphs, state machines, and dark factory.
What we really want is agentic execution on long-horizon tasks.
The gap is verification.
W/out verification, agents are vibing their way through tasks.
But there's no verification w/out discovery/identification first.
Robby retweeted
Anthropic has held the top 3 spots on @Arena's text leaderboard since April
Open-weight models like Kimi K3 and GLM-5.2 have broken through on coding boards. But when it comes to text-based tasks, Claude still dominates.
Yesterday, @Kimi_Moonshot's Kimi K3 open-weight debuted at #1 on @arena’s Frontend Code leaderboard, ahead of @AnthropicAI’s Fable 5 and @OpenAI’s GPT-5.6 Sol.
K3’s predecessor ranked #18 on the same leaderboard; one generation later, it's in pole position.
