CS PhD student at @Cornell | Prev: Student Researcher @GoogleDeepMind, Research Intern @MSFTResearch
Joined July 2020
- Tweets66
- Following219
- Followers255
- Likes274
Wasu Top Piriyakulkij retweeted
What makes a good representation for diffusion models?
Surprisingly, more information isn't always better.
In our new work, Purify Before You Align, we show that removing locally predictable information can substantially improve diffusion alignment.
🎥 A quick overview 👇
🧵 1/n
Evan always has simple yet interesting analogies for complex concepts. Go read his blog post!
wrote a small piece on agentic programming, evanthebouncy.substack.com/p… . It gives a few arguments on why we should intervene less and write less harnesses. give it a read if you got 2 minutes aha, it's fun :D
Wasu Top Piriyakulkij retweeted
Do coding agents understand the dynamics of the world well enough to reconstruct it from video?
We introduce 4DCodeBench to evaluate this ability through 4D inverse graphics.
4DCodeBench.com/
<🧵>
Wasu Top Piriyakulkij retweeted
Agentic real-to-sim is getting increasingly good at rebuilding scenes in a physics simulator. But looking at a scene does not reveal how glue cures, how water heats, or how a fan's wind pushes things. And what if the simulator cannot model these mechanisms at all?
We present EMPIRIC, an agent that learns a residual world model: a physics engine extended with code for the missing mechanisms.
It solves all 25 runs across five simulated domains, where coding-agent baselines solve 14 to 16, and on a real robot it learns wind and domino masses from two gusts.
basisresearch.github.io/empi…
[1/9]
Wasu Top Piriyakulkij retweeted
Predicting the answer to interventional "what if?" questions — the outcome of an action you never took — need a *mechanistic* model, not a curve fit. And you can only learn one by *experimenting*. Experiments are costly, so the real game is **data efficiency**.
Meet the Model Discovery Agent (MDA). 🧵
Wasu Top Piriyakulkij retweeted
Can AI agents act like scientists when reasoning about visual scenes? 🔬
Not just label what they see, but form explicit hypotheses, design experiments to test them, and revise their beliefs in light of the evidence. We built ZendoWorld to find out!
Wasu Top Piriyakulkij retweeted
How well can a model watch a short video of some physical dynamics and actually predict what happens next?
Introducing MPMWorlds: a new dataset and benchmark to evaluate how well models can reconstruct and extrapolate physical dynamics from video.
zzigak.github.io/mpmworlds/
🧵👇
(1/n)
Wasu Top Piriyakulkij retweeted
I agree that for agents (as opposed to making media slop for human consumption), symbolic world models that abstract away from pixels is key! I have a little paper on this (arxiv.org/abs/2602.02799) with @topwasu , @ellisk_kellis and @WLehrach. But we took a shortcut and assumed symbolic input (from OC Atari), so we could focus on learning temporally abstract WMs using code synthesis. Thus our approach didn't solve the signal-to-symbol problem, and relies heavily on pretrained LLMs, which feels a bit like cheating.
In general, once you leave the happy land of symbolic data (code, language, math), modeling becomes much harder, since you need to decide what loss function to use for training. If you can't afford to model every little detail, you can't use maximum likelihood (or some simple bound like ELBO), so you need some other way to decide how to "color the bits" (as @alemi likes to say).
New paper: Hierarchical Neural Options + Abstract World Model
w/ @WLehrach, @ellisk_kellis*, @sirbayes* (*equal advising)
Q1: How to move beyond the "one-step trap in AI research", building a temporally abstract model of the world?
Q2: How to acquire increasingly complex skills over time by building on existing ones?
A1+A2: We propose AgentOWL, an agent that jointly learns hierarchical neural options and an abstract world model
Limitations:
- Fully observed object centric state, where objects have names which are meaningful to the LLM
- The sequence of skills which agent needs to learn is currently manually provided, and must be short and sorted by difficulty
- Each game requires learning its own world model and policy, and the prompts are somewhat game specific
- Only evaluated on 3 (hard exploration) OC-Atari games.
But, exciting implications:
AgentOWL's successful results emphasize that option and world model learning are deeply intertwined, and that skills are fundamental to an agent’s understanding of its environment.
Work done during my internship at Google DeepMind
Arxiv link: arxiv.org/abs/2602.02799
I had a chance to talk about symbolic world models and PoE-World on Ndea's Abstract Synthesis podcast. It was fun—thanks Bryan and team!
On the pod: @topwasu from @ellisk_kellis' lab at @Cornell discusses his paper PoE-World.
We explore how symbolic world models can achieve strong generalization and sample efficiency by composing many small causal programs instead of learning a single monolithic model.
I'm presenting this paper at NeurIPS tomorrow (Dec 4) from 4.30-7.30pm at Exhibit hall C,D,E #508. Please come say hi!
New paper: World models + Program synthesis by @topwasu
1. World modeling on-the-fly by synthesizing programs w/ 4000+ lines of code
2. Learns new environments from minutes of experience
3. Positive score on Montezuma's Revenge
4. Compositional generalization to new environments
topwasu.github.io/poe-world
[1/n]
