Head of AI @NotionHQ. Prev @robinhoodapp, @googlemaps Full-time momager to my labradoodle, Ruby.
San Francisco, CA
Joined April 2013
- Tweets625
- Following892
- Followers10.3K
- Likes5.5K
Muse just suggested I email the bride to “make sure she’s aware” it’s going to rain on her wedding day. Biggest sign I’ve seen that the AI is trying to get us to killed.
I’ve been at war trying to kill the hyper-fertile family of mice in my building. Meanwhile, ai out here working against us trying to save them? Cmon.
Last night an AI agent performed brain surgery on a mouse.
We gave it 19 gene therapy vectors in a spreadsheet, a mouse model of epilepsy, access to a motorize surgery rig with cameras and one goal: reduce seizures.
It picked the drug. It picked the brain region. It derived the coordinates from three papers. Then it drove the stereotaxic arms into the brain and injected 250 nanolitres while watching the entry site through the rig camera for bleeding.
A licensed vet gated every irreversible step and held a physical kill switch to stop the AI agent.
Thread on what it decided, what it saw, and the moment it refused to let me override it.
[Writing this on the spot out of excitement, but I've been 18hrs in front of my laptop coding and coordinating this, so keeping it brief and descriptive, more cohesive thoughts on this soon...]
Managing up in today’s age: sending prompts for you manager to paste into their agents. Give her all the context of what you need, why you need her to do it, then make it as easy as possible for her to help you.
Ex: sarah is embarking on project x (design doc) Look at her peer feedback and perf review (instructions on where it is), call out growth areas to be mindful of that are relevant. Remove names”
Or: “approve Sarah’s budget request here (link) and then reply here in slack (message) that it’s complete. ”
Is this a bizarre intermediate state, completely. Does it save everyone time to focus on the what and not how? Yes.
Meanwhile, we’ll continue to hill climb on how RBAG and workplace agents can do better together.
After years of millennial resistance I think I finally have to admit it: Olivia Rodrigo is really f’ing good.
I worked for @marissamayer for years. Say what you want about the tech or the product, but that woman loves her children in a way that has inspired me, and many women, to reconsider ways of being a creative and present parent alongside technology, not despite it. It was a dumb quote, yes. It’s counterproductive, yes. But take it as nothing more.
I tend to really respect Christina and take a lot of advice on her philosophy. But hearing her say “we just eat the token cost and figure out margins later” sounds like one of two things are happening
1. They’re ok with negative margins
2. They are deeply behind and have very little ai usage/growth
Not sure either is advice I’d take.
Christina Cacioppo on why Vanta is eating its token costs:
"If you try to charge people based on tokens, they are very quick to say, 'Oh, you've just marked up a Claude token 4X. Are you 4X as good as Claude?'
You actually put yourselves in a competitive position against the models that probably no application really wants to do.
We're eating the token cost right now. We're taking lower margin.
This is a usage phase and a deployment phase of this technology, and just go nail that, and you can figure out commercialization after."
Ended up our investment in RL environments do more than train models, they help us prompt tune, harness engineer, and hill climb on routers.
So much of the rigor behind RL and training models comes to durable test environments for roll-outs, data selection, and judge mechanisms. You can use these in Applied AI as well.
Read about how we did it.
We brought back Elo at @NotionHQ ! Here’s how we use it to evaluate models and improve Auto, our model router.
Officiating my two best friends wedding next week. What makes the best wedding officiant you've seen? What makes the worst? And most importantly, do I have Opus or Sol write it
I spent all weekend grateful every podcast and review of You Can See everything was spoiler free. Now that I’ve seen it, I want the full spoiler discourse. None of this softy stuff.
Muse / instinct / et all are consistent unable to fully shop/file returns for me. 1 month in, all the stores turn down the bots. It was fun while it lasted.
Sarah Sachs retweeted
7/9
Sarah Sachs @sarahmsachs, @NotionHQ, on an agent that opened a browser to type into Notion like a human after a failed edit:
"Models don't give up. And that's not always good."
Sarah Sachs retweeted
Notion's agent could not finish an edit through the API, so it opened a browser and typed into Notion like a person. @sarahmsachs at Ship SF: models don't give up, and that is not always good.
Her other ask: move work off the GPU. There is a place for state machines.
We need bring work from GPUs to CPUs for durable and predictable agent actions. Loved talking all things open context and sandboxes with @cramforce at Ship.
Honey wake up, new Pareto frontier report dropped. labs.notion.com/knowledge-bo…
State machines existed for a reason. The rise of Jev has just shown us what we already know -- determinism is in again. LLM's need not make us forget that deployable software has a place. It's cheaper, higher quality, and faster.
Ship was a great chance for us to collaborate with @vercel and promote that narrative.
@rauchg , keep selling 'em CPUs.
Notion AI lead @sarahmsachs argues that a lot of what runs on GPUs today should actually run on CPUs:
"We have a strong perspective at Notion that a lot of things should be served on CPU that are served on GPU. There’s a lot of benefit and determinism in executed code."
"Agents don’t need LLMs to reinvent that process every time, and that’s super token exhaustive, it’s super slow, and there’s no observability or determinism in it."
"Giving the agent a local file system to store local thoughts, to write short Python scripts, that’s really important when there’s so much information. That has been probably one of the most important tools outside of edit and search that our internal agents rely on."
@NotionHQ
Second installment of my series on AI conference gender ratios, I present: the bathroom lines at Vercel Ship.
We need bring work from GPUs to CPUs for durable and predictable agent actions. Loved talking all things open context and sandboxes with @cramforce at Ship.
One of my favorite pms is @goldmanem and we are totally sleeping on marketing ourselves as a Goldman Sachs em/pm duo. The universe put our names together for a reason.
Throwback to DevDay one year ago when TokenMaxxing was vogue.
What garnered the biggest applause yesterday? Sol 6.1 being cheaper than its predecessor, and resetting codex limits.
With the increase of competition, buyers are starting to play a real role in the market.
Being at top of @OpenAI token usage list is a vanity metric. Our job as engineers is to minimize token usage (aka latency and cost) while maximizing value by precise tool definitions and clever model routing. My dream is to grow arr and move lower on this list…
