Evals @SpaceXAI. Making the most useful intelligence
Joined January 2017
- Tweets348
- Following782
- Followers2.2K
- Likes2.5K
Jonas Nelle retweeted
For the first time, Grok is better than Claude on Vending-Bench 2.
Impressive catch-up over the last few months.
Jonas Nelle retweeted
Introducing Grok Voice Transcribe 2.0. It’s the world’s most accurate speech transcription model.
Jonas Nelle retweeted
Introducing Projects, a new way of working in Cursor.
Rather than creating a chat for every task, you work with a coordinator agent in a single, persistent thread.
Like @bot, your agent is always on, proactively manages work with subagents, and improves over time.
AGI is no longer a meaningful goal.
SpaceXAI is the frontier model effort singularly focused on making the world’s most *useful* models.
Models that work with you like a (super-)human colleague, with judgement, creativity, and agency.
Jonas Nelle retweeted
Introducing Grok 4.6.
It delivers frontier intelligence and is a significant improvement over Grok 4.5 at the same price.
Jonas Nelle retweeted
Very nice but Grok 4.5 was left out for a reason!
54 @ $0.35
Jonas Nelle retweeted
In December, 1 in 10 of our merged PRs came from cloud agents.
Today, it’s 56%, as we use cloud agents to complete longer engineering tasks from start to finish.
We got here by giving agents their own cloud computers and letting them fix and improve their environments.
Jonas Nelle retweeted
Grok 4.5 is @kieranklaassen's 2nd most used model
holy shit, i need to update my priors
Jonas Nelle retweeted
I take it all back. We just got access to eval Grok 4.5 and it has landed above GPT-5.6-Sol and just shy of Opus for browser use.
Because cache input is expensive, the overall cost is only 10% cheaper than opus. Its overall a bit faster.
We have another opus-class model in the competition
Some new model release and my eval scores:
gpt-5.6-sol is a small improvement on browser tasks, but seems to consume many more tokens
Muse spark 1.1 outperforms gemini 3.5 flash at a cheaper price, but is not overall very good
Claude sonnet 5 is 5th best performance at higher cost than opus.
I am not very impressed with new models browser-use abilities. Maybe I will train my own instead
Jonas Nelle retweeted
Anyone calling Cursor an IDE, is completely out of touch with Cursor and likely hasn’t used it since the dark ages.
Cursor was an IDE last year, today, it’s one of the most powerful agentic coding platforms on the planet.
If you still think of it as an IDE, you should probably try Cursor, because I’d argue, you haven’t actually tried it yet.
Jonas Nelle retweeted
Recently met @srush_nlp and he started giving me an impromptu lecture on how targeted on-policy self-distillation works.
I asked him if I could record it on my iPhone.
The basic idea is this: if the model made a mistake at some point in the rollout (for example, calling a tool that doesn't exist), we want to discourage this specific error, but we don't want to just learn from the final reward, because it's a very noisy signal spread out over the whole trajectory.
So we have another model read this trajectory and figure where the error was made. It simply inserts some hint tokens to the part of the trajectory right above where the mistake was made.
Now with these injected hint tokens, have the model run a forward pass. You're not having to regenerate a new rollout - aka no new decode required.
The hint causes the model to assign lower probabilities to the error tokens. You then trains the original model to match these new probabilities, teaching it to downweight that specific mistake.
