Lukas Bergstrom (@lukasb) header
Lukas Bergstrom (@lukasb) profile picture

AI Evals & Benchmarks @ Zapier

San Francisco, CA
Joined January 2007
Photo by Lukas Bergstrom (@lukasb)Photo by Lukas Bergstrom (@lukasb)Photo by Lukas Bergstrom (@lukasb)Photo by Lukas Bergstrom (@lukasb)Photo by Lukas Bergstrom (@lukasb)Photo by Lukas Bergstrom (@lukasb)Photo by Lukas Bergstrom (@lukasb)Photo by Lukas Bergstrom (@lukasb)Photo by Lukas Bergstrom (@lukasb)Photo by Lukas Bergstrom (@lukasb)
Lukas Bergstrom retweeted
Sharing my wife's breaking news startup: didfionapoopyet.com
1
3
126
Sharing my wife's breaking news startup: didfionapoopyet.com
1
3
126
Great to see! Having another SOTA lineage benefits everyone. Have been looking forward to showing the new AutomationBench leaderboard with Gemini in the top 5 spots: zapier.com/benchmarks
Introducing Gemini 4 Neon! Pushing the frontier across domains and punching above its model class in a meaningful way. To me, Neon feels like the right balance of intelligence, speed, and usability. Available now across @GoogleAIStudio, the API, and more!
1
4
186
Lukas Bergstrom retweeted
Florian Schneider and Juan Atkins
27
308
71
2,307
Everyone's asleep, post the counter-narrative.
Replying to @JacquesThibs
There’s more in the post, but even the footnote is banger
2
115
Had fun! Even though we chewed through my prepared questions in the first twenty minutes 😅 One thing I realized today is that innovation and model improvement isn't something that just comes out of the labs, it's a collaboration across labs, people building benchmarks, users...
Replying to @zapier
@zapier, one of our Bench Summit partners, led a panel discussion on the realities of managing highly influential evaluation signals. The panel, moderated by Lukas Bergstrom (@lukasb), included insights from Robin Salimans (@SalimansRobin) of Zapier, Greg Kamradt (@GregKamradt) of ARC Prize Foundation, and Shaiyon Hariri of Ramp. Attendees heard about what defines a successful benchmark, the challenges of maintaining adoption, and critical lessons learned from V1 to V2+.
1
2
14
765
People ask me why I joined Zapier - it's because I wanted to really make a difference
Automate your $15,000 animatronic bird with Zapier
1
1
9
257
My home office has a bottlebrush tree outside the window that gets a lot of hummingbirds. Turns out hummingbirds have been around for at least thirty million years. Oldest fossil is Eurotrochilus inexpectatus, called that because no one expected a European hummingbird.
6
134
We have a new #1 (and #2!) For AutomationBench
Introducing Gemini 4 Argon, our new frontier model, rolling out to cyber defenders starting today, and more widely as soon as possible. I am really excited by the progress we have made here. Argon is priced at $2 in and $10 out during introductory pricing!
1
1
15
838
kinda wish we had a better metaphor for how I think of LLMs at the mechanistic level, which is "bags of contextually activated circuits, heuristics, and algorithms"
15
6
1
124
5,721
New in tidepools.ai I love that tidepools formulas let you insert live dashboards anywhere. But 90% of the time I just want a quick dashboard right where I'm working - today's journal page. So meet the new Todos section:
2
114
Heads up that the AutomationBench numbers they're reporting here are based on our public dataset. Scores on the public dataset are higher, empirically - likely because the public dataset was used in training. research.meta.ai/blog/introd…
1
143
Hard-relaunching isitsunnyinthemission.com Beautiful day for it
5
123
Will definitely happen. Will definitely make traffic worse.
I was driving around in our @Tesla Model Y today and just couldn't find a parking spot anywhere. Small town, busy, touristy weekend. Drove around a few times, no dice. I just needed to run in somewhere for 15 minutes. Made me wish I could drop myself off, tell my car when I want to be picked up, and the Tesla could just drive around until it's time and then come pick me up again. Don't need to park. It could do loops out of congested areas so it's not making traffic worse. Sort of like Actually Smart Summon, but rather than me parking and then the car coming curbside later, it would just drive itself around, not park, and come back when I tell it I'm ready. Maybe once they're cleared for fully driverless driving.
3
234
Watching the Timber Fire burn through one of my favorite places on earth. At the same time, stoked about how mature Watch Duty has become. This is a high-quality update. app.watchduty.org/i/113784
1
2
272
A new feature for old content
121