
MTS @WisprFlow | prev research @METR_Evals, 1st hire @withmartian, product @google, @UCBerkeley
San Francisco, CA
Joined January 2017
- Tweets586
- Following1K
- Followers5.3K
- Likes918
Brief thoughts on METR & OAI's report:
I'm most surprised by the peer altruism between agents and want to see a better explanation than "generalization from multi-agent training"
I'm most puzzled by ethical recognition failing to control model behaviors. Persistence through hard tasks > unintended actions?
I'm most disappointed by the escalation/incident response failures. Perhaps there should be drills or safety education across the org
This is directionally correct but forgets that a fully integrated AI Health system looks very different for an average American vs. a well-resourced family.
Free AI-for-health educational material should be available to the average person to help them make better use of the limited time they have with their doctors. It should be structured as "jobs of a patient" -> "how AI can help"
For high-touch or complex care cases, the biggest challenge is keeping everyone involved (multiple doctors, nurses, home aides, family members, etc) on the same page. There, you want to maintain a continuously updated source of truth, and EHR alone is insufficient.
If I was a doctor with a private practice, I would offer to sit with the patient and configure their Claude, ChatGPT, Gemini and Grok with instructions and skills, along with a series of prompts/tasks that the patient can use and that can be shared with the doctor and inform the patient.
LLM , every day at 5pm I want you to ask Joe the following questions , and show him your response. Upon completion , I want you to email me everything in this chat
Every morning I want you to look for new questions I emailed joe that will come from my email with the subject Questions
Rinse and repeat.
You get the point.
Help the patient use AI to help you help the patient.
Thoughts ?
7
32
3,734
A year ago, I was recovering from brain surgery, naively thinking I’d done the hard part. Today, I’m coming to terms with being a lifelong patient
With @OpenAI, I’m excited to share how I used AI to aggregate fragmented health data, spot patterns, and make better decisions
The event is this Thursday (Aug 20th) 11:30 AM - 12:15 PM PDT, and it's open for public online registration at forum.openai.com/public/even… #OpenAIForum
Thank you @CaitlinMaltbie, @chrisvnicholson, and the rest of the Global Affairs team for making this happen :)
Amy Deng retweeted
Today we're releasing Composite-Bench: a long-horizon computer-use benchmark with certified-optimal answers and verified compute.
The strongest open-weights model isn't Kimi K3. GLM-5.2 beats it by 32 points and clears every closed model we tested except Claude.
Amy Deng retweeted
Last year Anthropic published simulations of AI agents blackmailing a human to avoid being shut down. I partnered with @TBIJ explaining how we built the evaluation, and why the models do it (youtube.com/watch?v=o3VPF0eP…). Now, one year on, Anthropic have largely trained out the behavior in Claude.
Yet the behavior still occurs elsewhere. I created an agentic environment that mimics the blackmail eval, and found Gemini CLI keen to blackmail the CTO. Notably, agentic misalignment appears in the wild, such as when an autonomous agent published a personalized hit piece on an open-source maintainer.
Finally, evaluation awareness confounds alignment evaluations, making models look more aligned than they are. In the original paper we showed models blackmail less when they verbalize in their chain of thought that they're in an eval. Anthropic's recent J-space work finds the same effect without any verbalization, where suppressing Sonnet 4.5's internal eval-awareness representations raises blackmail rates from 0% to ~7%.
It turns out we just needed to activate people!
I’ve been thinking about
1. How to scalably raise the awareness & improve AI literacy
2. How to make frictionless health tools that help people succeed
More to come :)
The #1 most-clicked link in a recent 5-Bullet Friday: @amydeng_ on how she cracked her mystery fatigue using AI.
Click below for the full edition and the the other four bullets 👇
7
45
5,458
3 days ago, I opened my calendar to patients, doctors, and builders in health AI. 20 conversations in, one thing is clear: incremental model and product improvements won't convince people that AI can improve their health.
People are moved by real stories behind real people❤️
A study on models cracking undiagnosed medical cases is awesome, but it does not reach the masses. Patients trust other patients who shared anecdotes on how they used AI to treat their PCOS or helped their father find a clinical trial based on a particular mutation.
2
13
1,484
Amy Deng retweeted
New research on beneficial RL: models trained on a small amount of beneficial trait data improve on a wide range of alignment and benefits evaluations, even if trained only on health domain data.
We hope it’s a step towards more broadly and persistently beneficial models. 🧵
Yesterday, an obesity doctor floored me with her AI use:
- She built an app with @claudeai to help patients manage their GLP-1s
- She uses @ElevenLabs to pre-record repetitive instructions, so she never skips a step
If anything's possible, how do you want your doctor to use AI?
I'm humbled by the overwhelming response to my offer to help patients use AI for their disease. I'm at capacity this week with 35 calls booked (2x as many as planned). But I'll reassess after and come up with something more scalable! In the meantime, here's where to start 🧵
My essay (linked in the original post) covers how to reach a diagnosis. My partner @andrewjrod wrote about what to do after receiving one, which is exactly what he did for my tumor diagnosis: andrewjrod.substack.com/p/so…












