CS PhD @StanfordNLP | prev @princeton_NLP @SierraPlatform
New Jersey, USA
Joined April 2024
- Tweets186
- Following447
- Followers731
- Likes857
Pinned Tweet
Hello my 406 twitter followers. Hopping on to share two things:
1. I’ve started a CS PhD at Stanford! For the love of the game ofc.
2. I’m releasing impersona-env, a proof of concept to train personalized assistants by building an RL environment out of user models. Your assistant learns to help you by repeated asking a simulated version of you questions, probing its internal latent thinking and feedback, and updating its priors accordingly. I think this unlocks a new dimension to personalization beyond discrete fact recall. My personal assistant model now understands nuances of my preferences that even I didn’t know about.
Full blog: benshi34.github.io/#/blog/im…
It’s still a proof of concept and there’s still a ton of things to figure out in this space. It’s unclear to me how to build user model evals that generalize across users and downstream applications, how to best represent longitudinal personal data at training time // inference time, the interplay between population level and individual level preferences, the immense questions surrounding privacy and hillclimbing in decentralized data access regime… if you want to chat on anything in this area please dm me!
Ben Shi retweeted
I’m releasing OpenTTD-Bench, a benchmark for measuring frontier LLM agents on real-time, long-horizon tasks, based on a complex business simulation game OpenTTD, where models run competing transportation companies and must make the most amount of profit.
Ben Shi retweeted
Understanding AI’s impact on work starts with knowing what workers actually do.
Excited to share TaskNet 🥳 We use AI interviews to map tasks across 6 occupations, uncovering emerging tasks and overlooked work.
Amazing work by @jennjwang and the team!!!
Ben Shi retweeted
an example that folks often find surprising: codex (and others) scan your connectors / mcps *unprompted* in the background and you largely wouldn’t know when, what, or why
to catch this you need some form of intermediary infrastructure between your data and agents
Ben Shi retweeted
Jobs are modeled as bundles of tasks. But as jobs change, how do we keep those task lists up to date?
Workers often experience new responsibilities and tools before they reach their job descriptions.
We start by listening to them. 🧵
Ben Shi retweeted
I really like the vision behind this -- why haven't apple done this
To train user models / personal agents that best understand us, it helps to have access to all of our scattered digital footprint. For my first release in my PhD, I’m open-sourcing **Homecoming**, a desktop app that automatically aggregates all your personal context across all applications that you’ve accessed over your lifetime. It helps auto-request your data packages from over 2000 supported websites, and manages them locally in one place you can query, explore, and use for any downstream application. All your messages, emails, tiktok scrolling activity, goodreads, netflix, strava… even your 6th grade Wattpad account. You can even explore the features companies use to target ads at you.
This project was super fun to work on as I realized there is so much data about me that I didn’t even know existed. It’s a nice feeling to gain sovereignty over my data.
All data stays local, unless you choose otherwise. Lmk how it is!
Blog: benshi34.github.io/#/blog/ho…
Link to download: github.com/benshi34/homecomi…
Ben Shi retweeted
I'm fascinated. So many things I want to prompt myself on lol.
To train user models / personal agents that best understand us, it helps to have access to all of our scattered digital footprint. For my first release in my PhD, I’m open-sourcing **Homecoming**, a desktop app that automatically aggregates all your personal context across all applications that you’ve accessed over your lifetime. It helps auto-request your data packages from over 2000 supported websites, and manages them locally in one place you can query, explore, and use for any downstream application. All your messages, emails, tiktok scrolling activity, goodreads, netflix, strava… even your 6th grade Wattpad account. You can even explore the features companies use to target ads at you.
This project was super fun to work on as I realized there is so much data about me that I didn’t even know existed. It’s a nice feeling to gain sovereignty over my data.
All data stays local, unless you choose otherwise. Lmk how it is!
Blog: benshi34.github.io/#/blog/ho…
Link to download: github.com/benshi34/homecomi…
Ben Shi retweeted
I had tons of fun working on pretraining (& automating the pretraining) of Flint! It's a really good local model and a small preview of what we are cooking!
Introducing Flint, @Recursive_SI’s first model: a fast local coding model built with our automated research system, the Eureka Machine.
• Best-in-class fast local coding
• Up to 2x faster than comparable models: 125+ tokens/sec on a MacBook Pro, 150+ on RTX Spark
• Built from scratch in under 4 months
Private, unmetered intelligence on your own machine. Open weights available now on @huggingface, run it locally on your favorite framework or try it for free on @opencode or @pidotdev.
Ben Shi retweeted
user modeling is a key missing piece in making AI truly understand us and being useful, and making it sovereign is critical -- exciting work from @BenShi34!
To train user models / personal agents that best understand us, it helps to have access to all of our scattered digital footprint. For my first release in my PhD, I’m open-sourcing **Homecoming**, a desktop app that automatically aggregates all your personal context across all applications that you’ve accessed over your lifetime. It helps auto-request your data packages from over 2000 supported websites, and manages them locally in one place you can query, explore, and use for any downstream application. All your messages, emails, tiktok scrolling activity, goodreads, netflix, strava… even your 6th grade Wattpad account. You can even explore the features companies use to target ads at you.
This project was super fun to work on as I realized there is so much data about me that I didn’t even know existed. It’s a nice feeling to gain sovereignty over my data.
All data stays local, unless you choose otherwise. Lmk how it is!
Blog: benshi34.github.io/#/blog/ho…
Link to download: github.com/benshi34/homecomi…
Ben Shi retweeted
It’s coming home
To train user models / personal agents that best understand us, it helps to have access to all of our scattered digital footprint. For my first release in my PhD, I’m open-sourcing **Homecoming**, a desktop app that automatically aggregates all your personal context across all applications that you’ve accessed over your lifetime. It helps auto-request your data packages from over 2000 supported websites, and manages them locally in one place you can query, explore, and use for any downstream application. All your messages, emails, tiktok scrolling activity, goodreads, netflix, strava… even your 6th grade Wattpad account. You can even explore the features companies use to target ads at you.
This project was super fun to work on as I realized there is so much data about me that I didn’t even know existed. It’s a nice feeling to gain sovereignty over my data.
All data stays local, unless you choose otherwise. Lmk how it is!
Blog: benshi34.github.io/#/blog/ho…
Link to download: github.com/benshi34/homecomi…
To train user models / personal agents that best understand us, it helps to have access to all of our scattered digital footprint. For my first release in my PhD, I’m open-sourcing **Homecoming**, a desktop app that automatically aggregates all your personal context across all applications that you’ve accessed over your lifetime. It helps auto-request your data packages from over 2000 supported websites, and manages them locally in one place you can query, explore, and use for any downstream application. All your messages, emails, tiktok scrolling activity, goodreads, netflix, strava… even your 6th grade Wattpad account. You can even explore the features companies use to target ads at you.
This project was super fun to work on as I realized there is so much data about me that I didn’t even know existed. It’s a nice feeling to gain sovereignty over my data.
All data stays local, unless you choose otherwise. Lmk how it is!
Blog: benshi34.github.io/#/blog/ho…
Link to download: github.com/benshi34/homecomi…
Ben Shi retweeted
Language models already excel at math, but human growth remains essential: we need AI to teach people, not replace them!
Teaching is different from solving. As the Chinese principle “因材施教” suggests, preferences differ from person to person, and a good teacher should adapt their teaching to each learner.
We introduce Sherpa, a framework for training LLMs to teach adaptively.
Ben Shi retweeted
WildChat, LMSYS, ShareChat… each human-LLM chat dataset aims to capture "real-world" AI use. But do they paint the same picture?
Not quite. From user messages alone, a classifier can name a chat's source dataset, and user models trained on them inherit these signatures 🧵👇
Ben Shi retweeted
I'm on the academic job market this year!
Recently I've been working on scaling laws and human-centered interventions for building more socially grounded AI.
If you're attending #COLM2026, please drop me a DM or email. I'd love to chat!
Ben Shi retweeted
models can solve our hardest math and code problems, but they still struggle to understand user intent. strong user models will help us train better assistants, communicators, and collaborators
cool post by ben!
Hello my 406 twitter followers. Hopping on to share two things:
1. I’ve started a CS PhD at Stanford! For the love of the game ofc.
2. I’m releasing impersona-env, a proof of concept to train personalized assistants by building an RL environment out of user models. Your assistant learns to help you by repeated asking a simulated version of you questions, probing its internal latent thinking and feedback, and updating its priors accordingly. I think this unlocks a new dimension to personalization beyond discrete fact recall. My personal assistant model now understands nuances of my preferences that even I didn’t know about.
Full blog: benshi34.github.io/#/blog/im…
It’s still a proof of concept and there’s still a ton of things to figure out in this space. It’s unclear to me how to build user model evals that generalize across users and downstream applications, how to best represent longitudinal personal data at training time // inference time, the interplay between population level and individual level preferences, the immense questions surrounding privacy and hillclimbing in decentralized data access regime… if you want to chat on anything in this area please dm me!
Ben Shi retweeted
Personalized models mean personalized RL environments
Hello my 406 twitter followers. Hopping on to share two things:
1. I’ve started a CS PhD at Stanford! For the love of the game ofc.
2. I’m releasing impersona-env, a proof of concept to train personalized assistants by building an RL environment out of user models. Your assistant learns to help you by repeated asking a simulated version of you questions, probing its internal latent thinking and feedback, and updating its priors accordingly. I think this unlocks a new dimension to personalization beyond discrete fact recall. My personal assistant model now understands nuances of my preferences that even I didn’t know about.
Full blog: benshi34.github.io/#/blog/im…
It’s still a proof of concept and there’s still a ton of things to figure out in this space. It’s unclear to me how to build user model evals that generalize across users and downstream applications, how to best represent longitudinal personal data at training time // inference time, the interplay between population level and individual level preferences, the immense questions surrounding privacy and hillclimbing in decentralized data access regime… if you want to chat on anything in this area please dm me!
Thank you @modal for generously sponsoring the compute for this exploration.
And thank you @pranavreddy for playing league with me.