Assistant Professor of Biomedical Data Science and of Medicine at Stanford University
California, USA
Joined June 2010
- Tweets965
- Following446
- Followers1.3K
- Likes1.9K
Pinned Tweet
🚨 Postdoc openings @ Stanford (Fries Lab) 🚨
We build & rigorously evaluate healthcare foundation models from longitudinal EHRs, clinical text, and multimodal data.
🔬 Focus areas
• Multimodal & temporal modeling
• Clinical reasoning & grounding
• Benchmarking & evaluation
• Data-centric AI
• Human–AI teaming
💻 Work with massive real-world datasets (~4M patients; notes + imaging) and large-scale GPU compute (B200, H100, A100).
🎯 Applications: oncology, precision medicine, pediatrics
📅 Rolling review · Start Summer–Fall 2026
🔗 Details on how to apply: web.stanford.edu/~jfries/joi…
Jason Alan Fries retweeted
I've always believed that intelligence should come from simple, general building blocks that learn their own abstractions directly from raw data; in other words, the bitter lesson. I think we're finally seeing a real step towards AI that can learn from all data.
Language modeling on raw bytes is a microcosm of this ideal, and to me it's a central problem in AI: if we can't build LLMs that learn end-to-end from bytes, how will we handle the rest of the world's data that we understand even less?
Last year we introduced hierarchical networks (H-Nets) as a first step. Our new version finds up to 3-5x efficiency gains in controlled scaling law experiments, and at larger scale matches an open-source model trained on 11T tokens with ~10x less data and compute. The posts below walk through what we found and why we think it works.
Jason Alan Fries retweeted
The @SnorkelAI team did an audit of these findings using our internal QC platform.
Tl/dr: We agree with many of the findings, but ~55% of the fixes from @preferencemodel had their own issues with reward hacking or over-prescriptiveness.
Additionally: ~40% of the issues found were already tracked in the upstream repo - including 21 tasks with outstanding fix PRs!
Building datasets like this - and maintaining ambitious open benchmarks around them - is hard. We believe it requires open, collaborative review with community maintainers and relevant domain experts to get right.
@terminalbench was one of the first community efforts to build an open, continuous benchmark, and drive a vision of how OSS contributes to the rapidly advancing frontier of AI. We hope that the community will invest in working collaboratively with benchmark maintainers and task authors to keep this vision alive.
---
Some takeaways + details of our preliminary audit:
(1) We confirm 100% of the reward hacking / verification gap findings using our QC system. However, we find that 17/44 (39%) of the corrected tasks are just as or more reward hackable. Additionally, we don’t yet see evidence that these verification gaps meaningfully affected benchmark rollouts within CIs, though our analysis here is still ongoing.
(2) Our preliminary review raises questions about the underspecification findings for 22/41 (54%) of the flagged tasks, including 7 explicitly marked by task authors as intentional design choices. These cases warrant expert peer review and adjudication: balancing over/under-specification is a critical and tricky design decision - much of human intelligence concerns dealing correctly with reasonable and realistic underspecification - and we believe these corrections made to the tasks are likely too over-specified, reducing realism and difficulty relative to spec.
(3) Around 40% of the issues found were already tracked in the upstream repo (including 21 tasks with outstanding fix PRs)! OSS projects require community efforts to correct identified open issues, and ideally audits like these are done collaboratively with task authors and maintainers directly, in the traditional community spirit that has kept OSS alive for decades.
Jason Alan Fries retweeted
Strongly agree here. And a few quick thoughts:
- (1) Basic research will increasingly be about the empirical science of studying and understanding, not just improving model performance.
As AI models are increasingly *grown* as much as they are engineered, and as the complexity scale increasingly leads to emergent properties over discretely engineered ones, a lot of basic research is going to look more like classical empirical science - i.e. doing experiments, exploring, understanding, modeling, etc.
(One type of experiment: a benchmark, if done correctly; that's what we are attempting to fund and support through @SnorkelAI Open Benchmarks Grants!)
- (2) Basic research can prioritize *global optimization* (e.g. trying new architectures / algorithms / paradigms), which will always involve sacrificing local optimality at first.
These first, less optimal steps - which could take months, or years - are hard for *any* commercially-funded organization to do fully (especially once basic economic gravity returns in some areas of the space). This is the gap that basic research institutions have always- and will always- need to fill!
Also: these global optimization explorations do not always need 10K+ GPUs!! In fact, many will be selected specifically to explore new paradigms/approaches that are more compute efficient.
- (3) Basic research has many other beneficial externalities, e.g. supporting training/teaching, open dissemination of knowledge, etc.
Jason Alan Fries retweeted
I'm excited to announce @SnorkelAI's $350M Series E at $3.5B, led by @insightpartners and @S32_VC.
We've grown 18x+ in the last 12 months since launching our Data-as-a-Service offering, passing $375M ARRR this week.
As AI advances to superhuman capabilities, AI data & environment development must advance with it - and basic staffing and crowdsourcing approaches are not enough.
AI progress now requires deep research and technology work that combines human expertise with specialized AI in compounding ways. @SnorkelAI is building the RSI data engine and frontier data lab for this next phase.
We're honored to have the support of existing investors Addition, @lightspeedvp, @GreylockVC, @GVteam, P7, Factory, @WellsFargo, Walden Catalyst Ventures, and new investors @ThirdPointLLC, @MarchCPs, @BlumbergCapital, @AllegisCapital, @Frontlinevc, and @standard_vc.
–
@SnorkelAI started as a research project a decade ago at @StanfordAILab.
Our thesis was simple: AI progress would become increasingly data-centric – and therefore data development should be studied as a true research and technology problem, not just a staffing and crowdsourcing one.
Today, as AI capabilities verge on superhuman, building the data and environments to safely measure and train AI is becoming too hard for even the smartest human experts to do alone.
Only humans and AI agents, collaborating together in compounding ways, can meet the accelerating needs of the frontier, and keep humans in the driver’s seat of AI progress for decades to come.
At @SnorkelAI, we are building the data lab to define the shape of this new “Data 2.0” frontier, and the new paradigms of human-computer interaction needed to advance it.
Our key focus is building the RSI engine for data, where specialized AI models accelerate and improve human expert output, and in turn, scaled human supervision is used to continuously evaluate and improve these models – creating a powerful compounding loop to keep pace with an accelerating RSI frontier.
With this round of funding, we are also doubling down on our commitments to support data development for open benchmarking and evaluation (more news here soon!); an increasingly diverse ecosystem of general and specialized intelligence; and a path to safe, well-aligned AI built on robust training and evaluation data.
Data development will guide and drive the next stages of AI – and must do so in a human-centric, AI accelerated, open, diverse, and safe way. We are excited to support this mission in the next decade of research ahead at @SnorkelAI.
More thoughts here: snorkel.ai/blog/data-2-0-and…
Jason Alan Fries retweeted
This is notable. DeepSeek, a lab usually first to pioneer novel algorithms and architectures, is saying that at this point, the ROI of improving data quality far exceeds that of working on novel post-training algorithms.
I think this has already been true for some time for non-lab practitioners. If you're doing llm post-training, 80% of your effort should go into looking at your data.
This means:
- Hiring experts to dig through your RL tasks
- Sifting through rollouts and sft data by hand to remove suspicious samples. Make sure all tasks are actually passable.
- Making sure your data is diverse in both difficulty and category.
Jason Alan Fries retweeted
Don't be like these guys. Don't try to secretly scoop others when u hear about their potential success especially those with far less resources & power. Learn to give credit where it is due. Collaboration & healthy competition is the best way forward for science. Not this.
We’re sharing a solution to the Navier-Stokes Millennium Prize Problem, one of the deepest problems at the frontier of mathematics.
The proof was produced by a group of agents, using an OpenAI next-generation model significantly more capable than GPT-6 Astra.
The problem concerns whether the description of smooth three-dimensional fluid motion modeled by the Navier-Stokes equations can break down. It has remained unresolved for roughly 90 years.
Jason Alan Fries retweeted
OpenAI (& Anthropic) really need to release very clear legally binding statements ASAP that this kind of behavior is against their official policy. If not, it is going to be a legal disaster for science. Even more reason to favor open source models not controlled by companies.
This is legitimately terrible for OpenAI --- but also Anthropic --- and it relates to their work in drug discovery.
I read this as: prompts will be monitored and potentially acted on if we think there’s something scientifically and/or commercially valuable being disclosed.
Now, the big problem for Anthropic is that they are increasingly seen to be competing with their customers. Outside of SF, many people also think of OpenAI/ChatGPT and Anthropic/Claude as somewhat interchangeable companies and products --- aka, if OpenAI do something, surely Anthropic will do it too?
As so much IP relates to drug targets and people disclose these to Claude, I think we will see a lot of pharma companies stop using these tools or put up a lot of internal guardrails.
Jason Alan Fries retweeted
tristan buckmaster just published a statement that is, if accurate, one of the ugliest things i’ve read out of a frontier lab.
not because of the math. because of what happened after openai found out two people were about to beat them to the punch.
buckmaster and levent alpöge spent a year pushing the córdoba / martínez-zoroa program to smooth forcing. august 15 they got blowup for boussinesq and 3d euler. lean-verified august 22.
they were paying openai out of pocket and dumping every draft into private codex sessions.
the program is obscure. almost nobody else was on it. this was not “paste the clay problem into a chatbot.”
then this:
> thursday sept 3, buckmaster emails openai privately. rumors are flying. he is trying to stop a mess, not start one.
> they reply the same day: give us details so we can “avoid competing.” also, want free compute?
> sunday they suddenly need a call “at any point today.” sebastien bubeck gets on. levent is not invited.
> openai says an internal model has a ~100-page proof of forced navier–stokes blowup. fefferman options c and d. the exact route buckmaster and alpöge had quietly chosen.
> they show him a prompt and claim the model was “simply given the problem statement.” levent is told “very little human input.”
> that falls apart on the same call. a whole team had been on it. they started on the unforced problem, warmed the model up on easier equations including euler, and even the prompt they showed him was written by prompting codex.
> he asks when the first prompt went out. they stall. then agree: the past few days. after information about his work reached openai.
> he asks whether the model was trained on, or had access to, the private codex sessions where they put every draft of this project.
> answer: the model does not look up user data.
> he asks again. about training.
> no answer.
then it stops being a science story and becomes an hr story.
> offer 1: you post euler. we post navier–stokes the next day.
> offer 2: after you post euler, you alone write the navier–stokes paper, credit our model, and we cut levent.
> bubeck says twice he wants levent removed from authorship because he works at anthropic.
he declines both.
he says if they release it that way, he goes public.
the reply is not a scientific objection.
> “why would you ruin your career?”
> “if you don’t want me to be nice, then i don’t have to be nice.”
later, a text to levent proposing a one-on-one:
> “i don’t know if tristan is being fully rational right now.”
that is not how you treat a collaborator. that is how you split one.
openai got word two researchers were closing a path almost nobody else was on. they spun up a team on that exact path. they would not answer whether the researchers’ unpublished drafts in openai’s product were used. they tried to dictate the announcement order. they tried to remove a coauthor because he works at a competitor.
when that failed, someone reached for a career threat.
this is a lab finding out academics were about to publish, sprinting the same narrow program, refusing the data question, and trying to manage credit around a rival affiliation.
really slimy if even half of this holds.
what the fuck is happening.
holy shit.
tristan buckmaster is saying openai has a forced navier–stokes blowup via the córdoba / martínez-zoroa program.
and that it may have used drafts from his and levent alpöge’s private codex sessions.
that’s the córdoba / martínez-zoroa route. almost nobody else was on it.
he also says the first prompt went out after information about his and levent alpöge’s work reached openai. that “very little human input” was not true. that a whole team had already been set on easier problems, including euler. that even the prompt they showed him was written by prompting codex.
then he asked:
did the model have their private codex sessions.
no answer on training.
if any of that holds, openai didn’t just scoop a math result. they may have done it off two researchers’ unpublished drafts, then tried to cut the coauthor because he works at anthropic.
a millennium-problem-adjacent result just became employee drama before an ipo.
i don’t even know which part is crazier.
this is one of the slimiest things i’ve seen in this industry.
Jason Alan Fries retweeted
I had the honor of giving a keynote at the International Conference on Machine Learning in Seoul last week titled “What will be left for us to work on?” I addressed the widespread anxiety about how we should adapt as AI capabilities increase. I was thrilled by the talk’s reception, so I have made my slides available, annotated with a lightly edited transcript: cs.princeton.edu/~arvindn/ta…
I made three arguments. First, the "AI as Normal Technology" framework is a correct and useful as a way to think about AI’s impacts, unless and until there is some future discontinuity such as through recursive self-improvement. Second, even though we should take recursive self-improvement seriously, there is no milestone that companies might achieve in the lab that will suddenly put us all out of work. Third and finally, jobs of the future will be radically different, and a lot of adaptation will be needed. I shared my thinking about what this might look like and ended with a vision of human/AI “co-superintelligence”.
Jason Alan Fries retweeted
Heading over to Seoul for #ICML2026 soon! Excited to share lots of work from my group at @WisconsinCS and with @SnorkelAI: 4 main conference papers (one oral!) plus 3 fun workshop papers.
Reach out if you want to grab coffee!
🤖 Made with AI
Jason Alan Fries retweeted
One major factor distorting our perception of AI capabilities: benchmark development now lags behind model development for the first time in AI history.
In traditional AI/ML: The rate of benchmark advancement (i.e. labeling a small-to-mid sized dataset) exceeded that of model development - and so benchmarks gave a pretty useful view of frontier capabilities. This made them canonical measures of AI progress.
Today: it's very difficult to create benchmarks that properly measure *real world* environments, scenarios, and tasks at the jagged frontier of AI capabilities - which itself has become an exponentially bigger space to measure - and are robust to rapid overfitting. Benchmarks show near saturated performance - even though models still have real capability gaps in practice.
One more reason why accelerating the pace of benchmark development - and doing so with the full power of open, academic communities- is so important!
Jason Alan Fries retweeted
I find myself repeatedly explaining the difference between open-weight (DeepSeek), open-source (Olmo), open-development (Marin). Let's see if this restaurant analogy helps:
- Open-weight: food is made behind closed doors, server brings you the dish
- Open-source: food is made behind closed doors, server brings you the dish and the recipe
- Open-development: you see the chef make the dish in the kitchen (and can shout suggestions while its cooking)!
Jason Alan Fries retweeted
Hi friends, I’m joining @ChicagoBooth at @UChicago as an Assistant Professor of Applied AI!
I’ll keep working on AI + healthcare. I’m so grateful to my mentors @UMich for helping me find work I love (but after 12 years since undergrad here, it’s definitely time for me to go 😂)
Jason Alan Fries retweeted
Excited to head over to Rio for #ICLR2026. Presenting lots of work from my group at @WisconsinCS and with @SnorkelAI—reach out if you want to grab coffee!
Jason Alan Fries retweeted
These days, I'm much more excited about dataset releases than model releases. Models come and go and don't compose, whereas good datasets are more enduring and can be studied, used, revised to create better models more broadly. Excited about these 155K coding agent trajectories...just SFT'ing on this data improves SWE-bench Verified massively (23% -> 59.4%).
We’re open-sourcing CoderForge-Preview — 258K test-verified coding-agent trajectories (155K pass | 103K fail).
Fine-tuning Qwen3-32B on the passing subset boosts SWE-bench Verified: 23.0% → 59.4% pass@1, and it ranks #1 among open-data models ≤32B parameters.
Thread on the data generation pipeline 🧵
Jason Alan Fries retweeted
Meet Tabby 🐱- a simple architecture tweak that makes LLMs respect columns.
New TMLR paper (receiving a special J2C distinction—come check it out at ICLR!) led by @SonNicCr shows how a tiny modification turns pretrained LLMs into state-of-the-art tabular data generators.
🧵1/9
Jason Alan Fries retweeted
Agent skills are powerful.
But can they help us with agent orchestration and routing?
We show they can! 🎼
Introducing SkillOrchestra - a skill-aware alternative to RL-based agent orchestration:
💰 Higher efficiency
📉 Orders-of-magnitude lower training cost
🧠 More interpretable decisions
TL;DR: Stop training bigger orchestrators and start modeling skills.
Paper: arxiv.org/abs/2602.19672
[1/n]
