Sandhya (@sandhya) header
Sandhya (@sandhya) profile picture

Co-founder @Calibre_Labs | Applied AI research & consulting | Agents, AI Evals | prev EVP @amplitude_HQ | VC @khoslaventures @sequoia | @stanford @iitbombay

Joined May 2009
Loved the recent article on evals from @HamelHusain and @sh_reya in the @lennysan newsletter. One question we get everytime with PMs is what evals to write for long horizon agents that do a broad swath of work (what H/S call error discovery, similar to product discovery). Below is a great framework for how you can organize these discoveries as a product team: 1️⃣ Outcome - were the results great 2️⃣ Trajectory - was the process followed correct 3️⃣ Experience - did it feel right 4️⃣ Governance - did it follow critical rules You start with your product pov, but the only way to hone focus is to look at traces (trace analysis). Here's an example of an annotated trace for a local shopping assistant called Corner and how often the trace reveals flaws before they become obvious to the customer. claude.ai/artifact/PBvMFLueC…
6
15
1
46
4,773
easily #1 issue amongst teams we work with - no one is reading.. incredible amounts of alpha in being detail oriented rn!
I've spent the last 1.5 years working with the best AI startups globally. This distills my learnings on prompts. Engineers & builders, I hope it's useful 🤝
2
3
861
Hanging out with the original Claude today
5
361
chat am i the only one (and no this wasn't a computer use session) getting tired of the unending bugs
3
414
Increasingly hearing (and feeling) the word “addiction” being used by ai-pilled developers when describing our daily work.. mistaking output for progress and impact the way you mistake euphoria for happiness. I want my moments of friction to function as thinking breaks - forcing me to question whether something should exist rather than shipping. Building /= Shipping. The latter is a means to the former end. H/t @mitsuhiko
272
Replying to @cjc
I think the issue (no pun intended) is that your launch announcement says issue tracking is dead and this is the first real example of what's new in the changelog. It feels .. generated and not reviewed
1
4
826
I realized something else AI has changed about coding: you don't get stuck anymore. Programming used to be punctuated by episodes of extreme frustration, when a tricky bug ground things to a halt. That doesn't happen anymore.
1
2
274
Hi @claudeai can we do this to the topnav
1
2
383
Oh no why do the moltbots sound like linkedin influencers..
1
247
The only way to use twitter sanely is this: Following ordered by Recent. Else you are going to only see the most clickbaity content on the internet. And it's all going to be about Claude Code and the White House apparently.
2
2
329