Human guidance is part of the experimental setup. For agent research, recording when experts redirected the search and what they verified makes a result easier to interpret, reproduce, and turn into the next evaluation task.
We’re sharing new research demonstrating how AI agents can accelerate the science of stress-testing encryption.
Guided by humans, our models uncovered two major weaknesses in legacy 1024-bit RSA cryptosystems.
Our VP Science @Pushmeet explains more ↓ nitter.cf/pushmeet/status/211875…
Before scaling a robot dataset, test the task boundaries.
Can the policy still pick up the cup when its pose changes, the handle is partly hidden, or objects shift inside?
A pilot should map these conditions so the next collection batch has a clear purpose.
Image editing needs two scores: did the requested change happen, and did everything else stay intact?
Across repeated edits, track identity drift and unintended changes separately from visual appeal. A single preference score can hide the tradeoff.
Great to see a small team deliver a release like Ripple. For teams working with limited compute, task-level evals can help pinpoint gaps in data coverage. Knowing which examples to collect or improve next helps make each training run count.
