Research Scientist @ Collinear AI | PhD @ Cornell Defensive Cybersecurity
Sunnyvale
Joined July 2020
- Tweets67
- Following79
- Followers133
- Likes305
Gonzalo Gonzalez retweeted
I did say we'd be releasing our models periodically...introducing Gemini 4 Carbon, our next frontier model.
Following Argon's release just two weeks ago, Carbon shows the pace at which we’re pushing the frontier, it ranks #1 on the recently updated v5.0 of @ArtificialAnlys Intelligence Index at 59. It also leads across key cybersecurity evaluations scoring 88.2% on SEC-bench Pro for finding vulnerabilities, 54.8% on ExploitGym for exploitation, and 71.0% on CWE-bench for patching.
We’re seeing significant improvements in long-horizon tasks even compared to Argon, and we’ve strengthened frontier safeguards.
As we did with Argon, we’re going to roll it out gradually + responsibly. Some trusted cyber defenders will get access starting this week. We'll be expanding it to more partners through our Fairwind Program, ahead of wider availability soon, so hang tight!
With the ease of access to expert level offensive capabilities from agent, I expect many more irresponsible leaks like these from bad actors. It is crucial for enterprises to be up to date on the SoTA for defensive capabilities to mitigate damage to themselves.
This quoted post is unavailable.
Realistically though attackers will be constantly running their agents against your infrastructure and reacting to CVEs will be too slow. Proactively scanning your own systems with the strongest available agents can catch and patch vulnerabilities before attackers are aware.
At Collinear we’ve already gone through the effort of creating a map for which agent is best at detecting the most hidden vulnerabilities. Scanning your codebase with the best of each vulnerability class puts you one step ahead of attackers. Check out cwe-bench.com
Gonzalo Gonzalez retweeted
Exploration is hard for RL when training high-DoF dex hands ✋
X-Reset makes generalist object-centric RL easier by resetting to human hand-object states 🙋♀️
We mitigate the exploration problem, and train a single policy to manipulate a variety of objects to any goal pose 🧵
Gonzalo Gonzalez retweeted
Computer-use agents are limited by the interfaces humans design, typing a spreadsheet cell by cell when one line of code would do.
Can we let them discover their own interfaces through evolutionary search? Come to our poster and find out.
📍Continental Ballroom 6 at 2.30 PM at the Lifelong Agent (LLA) Workshop.
#COLM2026 #LLA, 2:30pm
openreview.net/forum?id=ctXj…
Gonzalo Gonzalez retweeted
Introducing MatFlowBench, our benchmark for LLM agents running materials science workflows.
Agents carry out first-principles calculations or design budget-limited measurement campaigns, then draw conclusions the resulting data support under the task’s stated physical assumptions.
Website: matflowbench.com
Blogpost: blog.collinear.ai/p/introduc…
Gonzalo Gonzalez retweeted
Stoked to share that cwe-bench is now the official benchmark for frontier labs for cyber capabilities.
congrats @sundarpichai and GDM team on a Pareto frontier cyber model 🚀
Lots of discussion out there about our next model(!), so I wanted to give an early look as soon as possible. Introducing Gemini 4 Argon!
It shows frontier performance in complex workflows, cyber defense and software engineering. Teams are using it extensively at Google, from coding to quantum computing, great feedback.
Here’s a look at the benchmarks:
There is quality control for vendor data that frontier labs buy to ensure their models don't go to shit; I'm sure you can add checks on model-generated data to avoid incentivizing killing humans (unless we want to benchmaxx that, in which case I'll share you samples)
To be clear, as we say in our latest Risk Report (nitter.cf/AnthropicAI/status/208…), I think the risk from present models is low. What I am worried about is superintelligence arising from recursive self-improvement, as we have said is happening faster than we thought (nitter.cf/AnthropicAI/status/206…).
Gonzalo Gonzalez retweeted
We’re also introducing Gemini 3.8 Flash Cyber, our most capable cybersecurity model. It shows frontier-level performance in discovering vulnerabilities and patching them at scale, with Flash-level speed & pricing.
That includes achieving 86.2% on the important CyberGym industry benchmark, plus 47.2% on CWE-Bench for patching. We saw a 70%+ success rate in discovering vulnerabilities across 20 programming languages on our internal benchmark.
Today we’re releasing CWE-bench: 100 held-out audit-and-patch tasks testing whether coding agents can defend real code.
The leading agent passes 47%. 18 tasks remain unsolved by any agent.
This means frontier models on our benchmark have de-correlated errors.
Blog: blog.collinear.ai/p/cwe-benc…
Gonzalo Gonzalez retweeted
Everyone has been talking about their agents escaping sandboxes since the July incident where two of @OpenAI models escaped a sandboxed cyber eval via a "read-only" registry proxy and ended up in @huggingface prod infra hunting for benchmark answer keys.
But rarely people talk about the environment or the simulated world around the agent.
I wrote a essay on it and propose a measurement for the environment's attack surface 🧵
Blog link: blog.collinear.ai/p/cybersec…
Gonzalo Gonzalez retweeted
Agent S3, the first computer-use agent that surpasses human-level performance in OSWorld v1 benchmark, has now been accepted to @TmlrOrg (Transactions on Machine Learning Research).
Congratulations to the @SimularAI team!
@chalo2000, vincent, @Richard_simular, jiachen, @xwang_lk
Stay tuned for what's next!
🚀 Introducing 𝐀𝐠𝐞𝐧𝐭 𝐒3, the most advanced computer-use agent, now 𝐚𝐩𝐩𝐫𝐨𝐚𝐜𝐡𝐢𝐧𝐠 𝐡𝐮𝐦𝐚𝐧-𝐥𝐞𝐯𝐞𝐥 𝐩𝐞𝐫𝐟𝐨𝐫𝐦𝐚𝐧𝐜𝐞🧠💻
Just one year ago, Agent S scored ~20% on OSWorld: SOTA then, but far from human 72%.
Today, Agent S3 reaches 6̳9̳.̳9̳%̳ (⬆10% over prior SOTA), nearly matching humans on computer use.
Few imagined this gap could close so fast, but with steady progress and 𝐟𝐢𝐫𝐬𝐭-𝐩𝐫𝐢𝐧𝐜𝐢𝐩𝐥𝐞 simplicity, @SimularAI did it in just one year.
🧪 The s̵e̵c̵r̵e̵t̵ 𝐨𝐩𝐞𝐧 𝐫𝐞𝐜𝐢𝐩𝐞:
→ 𝐁𝐞𝐡𝐚𝐯𝐢𝐨𝐫 𝐁𝐞𝐬𝐭-𝐨𝐟-𝐍 — scaling agents 𝘵𝘩𝘦 𝘳𝘪𝘨𝘩𝘵 𝘸𝘢𝘺
→ Simpler design, stronger baseline
→ 100% 𝐨𝐩𝐞𝐧 𝐬𝐨𝐮𝐫𝐜𝐞 (as always)
Learn more at simular.ai/articles/agent-s3 & 👇
Gonzalo Gonzalez retweeted
Our researchers @chalo2000 and @alckasoc just returned from @icmlconf with a lot to share!
A takeaways 🧵 (1/8)
First we presented our work, YC-Bench, a long-horizon planning and consistent execution benchmark!
Gonzalo Gonzalez retweeted
Can robots learn contact-rich tool use, like operating a pair of scissors✂️ or turning a screwdriver🪛, from human data?
Introducing REGRIND: a minimalist retargeting-guided RL recipe for dexterous manipulation.
🌐 yunhaifeng.com/REGRIND/
📄 Paper + code on the site!
🧵👇
Gonzalo Gonzalez retweeted
Teaching a robot shouldn't require humans to act like robots.
Human demonstrations contain valuable signal for robot manipulation, but they aren’t directly transferable to robots.
X-Diffusion learns from noisy human demonstrations while staying within the robot’s capabilities.
Gonzalo Gonzalez retweeted
Most “hard” problems are useless for training a model.
The useful ones sit in a narrow learnable region where the model fails sometimes and succeeds sometimes.
Here are some early results where we took 5 trivial and 5 impossible cybersecurity environments and had a model rewrite them toward that region (green) over a few rounds. 🧵
1/7
