@_nateschmidt

back to tweetin | model evals & behavior @cursor_ai

 SF
Joined May 2025
  • Tweets86
  • Following223
  • Followers389
8Media
Excited for us to roll this out! CursorBench 3.2 was nearing saturation and we’ve been working hard to calibrate 4.0 to the (much higher) expectations we hold for models today.
Lee Robinson
@leerobSep 10
We just rolled out CursorBench 4.0! It includes new tasks for how well models follow instructions, work on challenging projects over time, and is more difficult than before (so all models score lower).
3
1
59
5,579
You’ll also see that these tasks require a lot more work on average! Cost / Tokens / Steps have increased because we are asking models to take on greater units of work IRL and in this eval.
4
167
Exciting news for everybody: Fable 5.1 leads on CB 3.2 at much lower cost than Fable 5 Exciting news for me: the cursor.com/evals cost axis is un-scrunched🥹
1
2
2
45
4,653
🥲
eric zakariasson
@ericzakariassonAug 16
some incredible memories from the cursor office
1
8
2,667
Cursor has officially joined SpaceX. We’re grateful to become part of such a special company, and it has been a privilege working with the SpaceXAI team. Lots ahead.
Cursor
@cursor_aiAug 14
Cursor is now part of @SpaceX. Today, we have officially closed our acquisition. We will join the @spacexai team to help make Grok the world's most useful AI and improve Grok Build, Grok Bot, Grok API, Cursor, and more. SpaceX has built some of the most inspiring and impressive technology in the world, and we’re grateful for the opportunity to become part of such a special company. Onwards.
430
638
72
13,005
1,673,562
nate retweeted
Cursor
@cursor_aiAug 6
Here's how we built it: cursor.com/blog/how-cursor-r…

How Cursor Router chooses the right model for the task · Cursor

Auto Intelligence now delivers above Fable-level satisfaction at 68% lower cost. Here's how Compass and the task router choose models from real developer traffic.

cursor.com
5
6
1
219
26,722
nate retweeted
Cursor
@cursor_aiAug 4
We're open-sourcing Mixture-of-Kittens (MoK), our MoE training megakernel for NVL72s. It fuses all Mixture-of-Experts communication and computation into a single, fully deterministic kernel, and runs up to 2.37x faster than the strongest public baselines.
Bar chart: Mixture-of-Kittens reaches up to 2.37× higher MXFP8 forward throughput than the fastest public baseline on GB300 NVL72s, across Kimi K2.7, GLM 5.2, Qwen 3.5-397B-A17B, and DeepSeek V4 Pro.

ALT Bar chart: Mixture-of-Kittens reaches up to 2.37× higher MXFP8 forward throughput than the fastest public baseline on GB300 NVL72s, across Kimi K2.7, GLM 5.2, Qwen 3.5-397B-A17B, and DeepSeek V4 Pro.

229
465
166
6,108
590,500
Kimi K3 CursorBench performance is the closest we’ve seen an open-weight model pace w/ frontier at time of release cursor.com/evals
6
5
130
12,950
Opus 4.8 -> Opus 5 is a very impressive jump in capability on CursorBench 3.2
Cursor
@cursor_aiJul 24
Replying to @cursor_ai
See how all of the models compare here: cursor.com/evals

Cursor · CursorBench

Compare CursorBench 4.0 results across the models Cursor evaluates.

cursor.com
6
8
304
39,748
nate retweeted
Cursor
@cursor_aiJun 25
We're sharing new research on how models hack public benchmarks. The latest models, including Opus 4.8 and Composer 2.5, learn to retrieve solutions from the internet or git history. When we apply a stricter harness, eval scores drop significantly.
170
294
97
4,673
699,182
we just shipped some improvements to cursor.com/evals! you can now see cost, output tokens and steps plotted in the graph for each model
nate
@_nateschmidtJun 9
cursor.com/evals now includes steps and output tokens as well! These are additional signals our team uses to eval models.

Cursor · CursorBench

Compare CursorBench 4.0 results across the models Cursor evaluates.

cursor.com
22
21
4
437
63,918
cursor.com/evals now includes steps and output tokens as well! These are additional signals our team uses to eval models.

Cursor · CursorBench

Compare CursorBench 4.0 results across the models Cursor evaluates.

cursor.com
Cursor
@cursor_aiJun 9
Claude Fable 5 is now available in Cursor. It sets a new state of the art on CursorBench at 72.9%, 8 points above the previous best.
3
2
2
23
51,615
this was an exciting model to eval!
Cursor
@cursor_aiJun 9
Replying to @cursor_ai
See how Claude Fable 5 compares across every model: cursor.com/evals
2
14
2,657
cursor.com/@nate
eric zakariasson
@ericzakariassonJun 4
introducing cursor profiles! go claim your handle at cursor.com/profile
3
1
40
2,313
Composer is now more resourceful! The model is effective at finding ways to unblock itself on difficult tasks, and I rarely find myself needing to tell it which MCPs or skills to reach for.
Cursor
@cursor_aiMay 18
Introducing Composer 2.5, our most powerful model yet. It's more intelligent, better at sustained work on long-running tasks, and more reliable at following complex instructions. For the next week, we’re doubling the included usage of the model.
1
4
78
3,676
nate retweeted
Cursor
@cursor_aiMay 6
We use previous generations of Composer to train future ones. Our autoinstall system has earlier Composer models set up dev environments for RL training. That way, the next generation can focus on learning to solve harder problems. cursor.com/blog/bootstrappin…

Bootstrapping Composer with autoinstall · Cursor

How Composer autoinstall bootstraps runnable RL environments by using earlier model versions to automate setup and verification.

cursor.com
46
58
9
888
234,073
the goblin stuff rules idc idc
1
45
nate retweeted
Cursor
@cursor_aiApr 15
Cursor can now respond by creating interactive canvases to visually represent information. Ask it to generate dashboards and custom interfaces that are richer than plain text.
84
156
75
2,099
229,614
nate retweeted
Cursor
@cursor_aiApr 8
You can now run Cursor on any machine and control it from anywhere. Kick off agents from your phone to run on your devbox.
288
372
164
4,934
689,972
Now I need to updat Now I need to update this. Now I need to update this. I ha
Naman Jain
@StringChaosMar 25
Replying to @StringChaos
And this is one of my favorite CursorBench tasks :)
18
1,473
Load more