
research @openai. prev @tesla_optimus, phd @stanfordailab, basc @uoft engsci
馃嚬馃嚰馃嚚馃嚘馃嚭馃嚫
Joined September 2012
- Tweets166
- Following454
- Followers1.4K
- Likes1.2K
Pinned Tweet
i just joined @openai to work on personal agi. this release is a glimpse of why i鈥檓 so excited
yesterday鈥檚 math results show just how powerful frontier models are now. there鈥檚 an immense opportunity to transduce that intelligence into utility and joy for everyone
leaving @tesla_optimus and robotics was bittersweet, but the opportunity to shape how ai can benefit 1.2b+ people today was too hard to pass up
besides, we鈥檒l all be working on robotics soon enough :)
that's my labmate 馃コ
insanely impressive capabilities from NO ROBOT DATA
Today, we present a step-change in robotic AI @sundayrobotics.
Introducing ACT-1: A frontier robot foundation model trained on zero robot data.
- Ultra long-horizon tasks
- Zero-shot generalization
- Advanced dexterity
馃У->
1
27
3,768
Congrats Chuning and collaborators! Really nice to see work on doing joint generative modelling over all (or at least more) of our available robot data.
On a personal note, I remember pushing for this high-level approach to be articulated in the robotics section of the foundation models paper, so a small pat on the back for past me (and maybe a cookie).
Scaling imitation learning has been bottlenecked by the need for high-quality robot data, which are expensive to collect. But are we utilizing existing data to the fullest extent? A thread (1/11)
3
1,249
FlowMo is a SOTA image tokenizer based on diffusion autoencoders -- no convolutions, adversarial training, or distillation! Personally, I expect to see similar approaches from frontier labs soon: diffusion is much more scalable than convnet GANs.
Modern generative models of images and videos rely on tokenizers. Can we build a state-of-the-art discrete image tokenizer with a diffusion autoencoder? Yes! I鈥檓 excited to share FlowMo, with @kylehkhsu, @jcjohnss, @drfeifei, @jiajunwu_cs. A thread 馃У:
1
7
1,304
Check out our work on LLM personalization led by Anikait and Sheryl!
Personally (hehe), it's really cool to see how ideas in automated few-shot task construction I explored way back before my PhD remain applicable in the era of foundation models.
Personalization in LLMs is crucial for meeting diverse user needs, yet collecting real-world preferences at scale remains a significant challenge. Introducing FSPO, a simple framework leveraging synthetic preference data to adapt new users with meta-learning for open-ended QA! 馃У
5
858
Incredibly saddened to hear of @FelixHill84's passing. His work and talk on "Environmental drivers of systematicity and generalization in a situated agent" (arxiv.org/abs/1910.00571) directly inspired my first successful PhD project.
Rest in peace, Felix.
9
1,671
Kyle Hsu retweeted
Is it possible to obtain zero-shot generalization of vision-based RL agents without data augmentation, task-centric representations, etc? Yes! by disentangling the latent space and, you might鈥檝e heard about this given the latest news, associative memory i.e. Hopfield models!馃У
馃毃 NEW state-of-the-art model for unsupervised disentanglement 馃毃
[1/n] Tripod melds three complementary methods for disentangled representation learning that each target a separate component of an autoencoder.
[14/n] Jubayer and I will be presenting this work at ICML this Thursday, 11:30 am to 1 pm local time, Hall C 4-9 #409. Come chat about disentanglement!
1
5
939
[n/n] For more details, including implementation, fully-fledged background, derivations, decoded latent intervention visualizations, etc., see
code: github.com/kylehkhsu/tripod
paper: arxiv.org/abs/2404.10282











