AI for Scientific Discovery | Stanford
Joined October 2013
- Tweets78
- Following317
- Followers332
- Likes1.5K
Have noticed Codex being kinda lazy lately. Hit run on an open-ended task last night and it spent a whooping 12m to give a mediocre solution. Opus 5.5 in CC w same prompt works for hours. Can we config persistence like we can currently scale effort? Thoughts @thsottiaux ?
Owen Queen retweeted
I improved NanoGPT Speedrun from 39.9s → 6.7s (-83.2%), more than every record since 2025 combined (-81.4%)!
The web is full of rare, repeated content that the LM can’t memorize in time. I solve this with retrieval-augmented n-gram hints🧵
Owen Queen retweeted
AI is now taking on some of the hardest open problems that scientists have posed. But research is broader than solving problems. Can AI also propose research questions that experts find genuinely interesting?
We built After42 to find out. Experts judged 45 of its 135 final questions genuinely interesting, across mathematics, statistics, theoretical computer science, and physics, with wide variation across subfields.
The key idea: develop questions by trying to solve them.
We also asked whether AI can judge which questions are interesting. We find that model judges' top choices align with expert consensus but do not consistently capture individual experts' preferences.
We release the expert-assessed questions as an open resource for researchers to discuss, refine, and pursue, as a step toward integrating AI into the broader research enterprise.
after42.org
Owen Queen retweeted
Your LLM agent has to think out loud, token by token, before it can search, run code, or act in the world.
We humans don’t. We ponder in silence and speak when it matters.
Human language was never meant to be the limit of machine thought.
Introducing 🌗Hybrid Reasoning Agents
- Think in latent space, act through text
- Two modes, one unified multi-turn flow
- No black box, fully readable latent thoughts
- Any LLM can become a HybridAgent
Owen Queen retweeted
AI-generated scientific output is scaling beyond available human review capacity. Scaling scientific discovery requires scaling verification.
We introduce OSVerify, a framework for one-sided verification: rejecting as many invalid solutions as possible while preserving rare valid ones for expert review.
We used OSVerify to referee 11 recent OpenAI math preprints, identifying 5 major and 13 minor issues, all verified by human experts.
Our key insight: checking a proposed error can be easier than judging an entire solution. Instead of directly judging solution correctness, OSVerify decouples verification into error proposal (searching for potential errors) and error validation (checking whether they are real and substantial).
On competition and research mathematical benchmarks, OSVerify reduces invalid solutions requiring expert review by 46% on average compared to voting under the same low-FNR constraint, or 57% with an RL-trained validator.
Verification can also guide discovery. OSVerify can be integrated into existing scientific discovery workflows. Paired with an agentic math solver, it helped resolve 3 open conjectures in the Langlands program and contributed to resolving the Manin constant conjecture. All proofs were confirmed by mathematicians.
Our broader goal is to provide verification-based evaluation signals for scientific discovery, enabling test-time scaling and self-improvement even when reliable automatic evaluators are unavailable.
Project: osverify.github.io/
Paper: osverify.github.io/assets/pa…
GitHub: github.com/OSVerify
[1/n]
Owen Queen retweeted
🧵Thread (1/9): People increasingly turn to AI for personal advice, and LLMs tell them what they want to hear, endorsing the user far more often than humans do.
The problem: people become less willing to repair their relationships after conflicts.
We propose PlurPO, a method to construct a pluralistic preference dataset to post-train models to mitigate this social sycophancy.
Owen Queen retweeted
Students sometimes ask me if it still makes sense, in this accelerating age, to pursue a PhD in AI.
Perhaps counterintuitively, I think it's a great time to do so.
I wrote up some thoughts on this here: web.mit.edu/phillipi/www/wri…
Owen Queen retweeted
The recent breakthrough in Navier-Stokes has garnered a lot of attention to the new possibilities that emerge when agents work together.
We have been interested in this question for a while. How can a team of agents achieve more than agents working alone?
I'm excited to finally share our new work, "Self-Organizing Agent Teams Learn to Reason Together" 🧵
Owen Queen retweeted
Thrilled that #paper2agent is published in @nature today!
Scientific knowledge is traditionally stored in passive papers. Paper2Agent transforms papers into virtual authors that answer questions, apply its methods and collaborate w/ other paper agents to make new discoveries 🧵
Owen Queen retweeted
New work I'm very excited about: Physics of Agents!🤖
As AI agents become more prevalent, they'll interact and influence one another.
Can we predict the collective behavior that emerges?
Surprisingly, their dynamics follow compact, predictive laws of statistical physics🧵
Owen Queen retweeted
Introducing "The Agentic Garden of Forking Paths"🌱
Give two AI agents the same data, the same question, and the same instruction to "analyze it rigorously"; change only one paragraph's persona stating a prior belief, and they reach opposing scientific conclusions:
🧵👇
Owen Queen retweeted
🧵Thread (1/7): Jobs are bundles of tasks. Technology changes work by changing tasks.
But how you get from tasks to jobs matters when doing job-level analysis — and existing approaches get it wrong. We posit that weighting tasks by working time share is a strong choice of task weights.
🌵 We publish a comprehensive set of time share estimates for nearly all 18k ONET tasks and 1k ONET jobs.
🌿We then look at the share of working time across jobs that is exposed to AI.
Owen Queen retweeted
1/ Are tokens the right action space for understanding how reasoning models solve problems?
We found something higher-level: their chains of thought decompose into recurring problem-solving moves shared across models.
New paper with @StanfordAILab: ReasonOps. 🧵
1/ 🧠 Can an AI overthink a problem? Turns out yes — and we can measure it.
New paper: ReasonOps. We built a way to read the "thinking" inside reasoning models and found a hidden structure shared across every model we tested. 🧵
6/ Thanks to my awesome coauthor Daniel Lee and advisor @james_y_zou
Paper: arxiv.org/abs/2605.29192
GitHub: github.com/lee-dan/ReasonOps
5/ ✅ How a model reasons predicts whether it's right — often before it even finishes.
Annotation-free and unsupervised, so it scales to any model.
4/ 🔍 Every model has a "reasoning fingerprint."
The pattern of moves a model uses is so distinctive you can identify which model wrote a trace from its reasoning style alone — no other information needed.
3/ 🔁 Overthinking is real.
Reflective moves help on hard problems — but actively hurt accuracy on easy ones. Sometimes the smartest thing a model can do is commit to an answer.
