
PhD Student @kaist_ai | Research @RLWRLD_ai
Joined April 2023
- Tweets27
- Following122
- Followers80
- Likes197
Pinned Tweet
Excited to introduce FRED, a WAM framework improving zero-shot generalization and human-to-robot transfer! ๐ค
We view VDM as future representation generators and improve them via self-distillation and video conditioning.
๐ huiwon-jang.github.io/fred
๐ค huggingface.co/FRED-AGI-LAB
A recent line of WAMs generates actions from VDM hidden states of a single denoising step, yet their quality gets little attention.
๐ We introduce FRED to improve them via self-distillation, with large gains on real robot experiments.
๐ huiwon-jang.github.io/FRED/
๐งต [1/7]
7
212
Huiwon Jang retweeted
๐ค Which robot policy checkpoint should you deploy? Lower validation loss can point you to the wrong one.
Introducing ๐ฅ๐๐๐: ๐๐ฎ๐น๐ถ๐ฏ๐ฟ๐ฎ๐๐ฒ๐ฑ ๐๐๐บ๐๐น๐ฎ๐๐ถ๐๐ฒ ๐ฆ๐ฐ๐ผ๐ฟ๐ถ๐ป๐ด for rollout-free checkpoint selection.
RACE ranks checkpoints from a fixed set of successful reference trajectories, without running each candidate on the robot.
๐งช Simulation: stays reliable when references shift from demos to noisy rollouts, while validation loss turns negative
๐ฆพ Real robots: 1,800 evaluations on 6 tasks. RACE is highly correlated with real deployment success rate for both Diffusion Policy and ฯ0.5-DROID, while baselines failed.
๐ฌ 60-second overview below
๐ changyeon.site/race/
๐งต 1/8
Huiwon Jang retweeted
Robots have learned from robot demos. But can they learn new tasks from human experience alone?
๐ค CoRL 2026 Scaling H2R Workshop - Oral Presentation
Introducing H2RB: Human-to-Robot Bridge. We bridge perception and action gaps to transfer new skills and scale human experience across robot embodiments. ๐งต
๐ Paper: arxiv.org/abs/2611.04391
๐ Project: junhyeong-park.me/h2rb/
๐ค HF Paper: huggingface.co/papers/2611.0โฆ
๐ฅ๏ธ Code: github.com/jun-brro/h2rb-codโฆ
Huiwon Jang retweeted
๐ค Improving robot foundation models with real-world RL, then continually carrying that experience forward across tasks.
Introducing ๐๐๐๐ฅ๐จ๐: ๐ฐ๐ผ๐ป๐๐ถ๐ป๐๐ฎ๐น ๐ผ๐ณ๐ณ-๐ฝ๐ผ๐น๐ถ๐ฐ๐ ๐ฅ๐ for robot foundation models.
Our real-world policies learn through physical robot interaction, starting from a small set of demos.
We inherit the actorโcritic across tasks and distill earlier behavior, so experience accumulates while prior skills remain.
๐ป ๐๐ผ๐ ๐ฎ๐ฑ๐ฑ๐ถ๐๐ถ๐ผ๐ป๐ฎ๐น ๐๐ฃ๐จ ๐๐ฟ๐ฎ๐ถ๐ป๐ถ๐ป๐ด ๐ผ๐๐ฒ๐ฟ๐ต๐ฒ๐ฎ๐ฑ: RL updates a compact actorโcritic while the foundation model stays frozen.
โก Subsequent tasks reach 95% success in about ๐ฐโ๐ญ๐ฎ ๐บ๐ถ๐ป๐๐๐ฒ๐ ๐ผ๐ณ ๐ฟ๐ผ๐ฏ๐ผ๐ ๐ถ๐ป๐๐ฒ๐ฟ๐ฎ๐ฐ๐๐ถ๐ผ๐ป.
On real robots:
๐ Up to 68.5% less robot interaction vs. single-task RL
๐ฏ 98.3% final prior-task success
๐ฌ A 60-second overview below.
๐ accrue-rl.github.io/
๐งต 1/6
Huiwon Jang retweeted
Introducing SQAM, Q-Learning with Scalar Adjoint Matching! ๐ Off-policy RL for flow policies, without a single VJP through the policy. Bigger gains on the hardest tasks, at a lower cost per update. ๐งต [1/8]
Huiwon Jang retweeted
๐1st place in the AI Scientist track at Ralphthon @ ICML2026 โ $10,000 in OpenAI credits.
@happyhappy_jun & @WoominSong from @KAIST_AI
We've open-sourced all of it: harness, code, the full log of research traces.
See our post: byungjunyoon.ai/writing-drivโฆ
How we did it ๐งต
๐ผ Can we turn music into readable code?
Excited to share Decomposer, a post-training framework for symbolic music decompilation!
Given MIDI, Decomposer recovers an executable program in Strudel that reconstructs the music while exposing patterns, harmony, rhythm, and voices as editable code.
๐งต [1/5]
Huiwon Jang retweeted
Can MLLMs actually track what's happening in a video?
Introducing VSTAT ๐ฏ, our new benchmark for visual state tracking.
The tasks are simple: count cups, read typed words, count page flips. Humans solve them easily. MLLMs don't.
vision-x-nyu.github.io/vstatโฆ
๐งต [1/11]
Huiwon Jang retweeted
Introducing TRQAM! Internalizing a KL trust region inside the sampling SDE stabilizes off-policy RL fine-tuning of pretrained flow policies. With TRQAM, we lift offline RL success on 50 OGBench tasks from 46% to 68%. ๐งต [1/8]
yonghdong.github.io/blog/trqโฆ
Huiwon Jang retweeted
RLDX-1 took the first place "officially" in the RoboCasa365 benchmark!
robocasa.ai/leaderboard.html
Tech report: arxiv.org/abs/2605.03269
Excited to share RLDX-1, a robot foundation model for human-like dexterous manipulation in the real world!
It integrates versatile intelligence with functional capabilities, outperforming ฯโ.โ
and GR00T N1.6 across simulation and real-robot tasks.
๐rlwrld.ai/rldx-1
(1/12) Hi, we are #RLWRLD (ReaL WoRLD).
RLDX-1 is live. Dexterity is intelligence, and it lives in #RLDX (RealDex).
A dexterity-first foundation model from for robot hands that builds muscle memory through motion, history, and contact.
- Across 10 dexterous real-world tasks on the ALLEX humanoid and DROID setup, RLDX-1 ~2ร outperforms ฯโ.โ
and GR00T N1.6.
- SOTA on 8 simulation benchmarks in LIBERO, SIMPLER, and RoboCasa.
Everything ships today:
training and inference code, the pre-trained model, mid-trained checkpoints, and a fine-tuned checkpoint for every reported benchmark.
LoRA recipes for parameter-efficient adaptation to your robot. Supports LeRobot v2.1 datasets. A one-line CLI to toggle motion / memory / physics modules per embodiment.
Full technical report with ablations.
Not just our cool demos. We document the architecture, data, and design decisions behind every result. Bring your own robot. Train and deploy RLDX-1 on it.
๐งต What's inside RLDX-1, in 12 posts.
๐ rlwrld.ai/rldx-1
๐ arxiv.org/abs/2605.03269
๐ป github.com/RLWRLD/RLDX-1
๐ค huggingface.co/collections/Rโฆ
Huiwon Jang retweeted
RLDX-1: A Vision-Language-Action model for human-like dexterous manipulation
Built on the Multi-Stream Action Transformer, it unifies heterogeneous modalities with motion awareness, long-term memory, and physical sensingโachieving 86.8% on ALLEX tasks versus ~40% for ฯ0.5 and GR00T N1.6.
A Design Space for Live Music Agents ๐ท๐น๐ฅ #CHI2026
What does it take for AI to truly jam with you? We surveyed 184 live music agents across AI, HCI, and Computer Music fields to map the design space, and where it's headed.
๐๏ธ Talk: Fri Apr 17, 12:15PM ยท P1 Room 132
๐ Paper: arxiv.org/abs/2602.05064
๐ Interactive demo: live-music-agents.github.io/
Huiwon Jang retweeted
Catastrophic forgetting has long been a challenge in continual learning.
However, our new study found that pretrained Vision-Language-Action (VLA) models are surprisingly resistant to forgetting!
Zero forgetting, or even positive backward transfer, is possible with simple experience replay.
arxiv.org/abs/2603.03818
Huiwon Jang retweeted
Excited to share Adaptive Low-Pass Guidance (ALG): a simple training-free, drop-in fix that brings dynamic motion back to Image-to-Video models! Demo videos, paper, & code below!
choi403.github.io/ALG (๐งต 1/7)
Huiwon Jang retweeted
Meet MELD: a masked diffusion model (MDMs) designed for de novo molecule generation. MELD assigns per-element learnable noise schedule that tailors noise at the atom & bond level to avoid state-clashing problem. With MELD we achieve state-of-the-art property alignment in multi-conditioned generation and a >70% boost in chemical validity.
Dive in ๐ holymollyhao.github.io/MELD/ [1/7]
Huiwon Jang retweeted
Excited to present FastTD3: a simple, fast, and capable off-policy RL algorithm for humanoid control -- with an open-source code to run your own humanoid RL experiments in no time!
Thread below ๐งต
Huiwon Jang retweeted
Exicited to share my internship project ๐ Great experience to work with awesome people.
If you are interested in improving LLM pretraining, concepts, and distillation, please take a look!
๐ฅฅ๐ช๏ธ Introducing CoCoMix - a LLM pretraining framework that predicts concepts and mixes them into its hidden state to improve next token prediction.
๐ More sample-efficient and outperforms next token prediction, knowledge distillation, and inserting pause tokens.
๐ฌBoosts interpretability and steerability by analyzing and modifying the predicted concepts.
๐: arxiv.org/abs/2502.08524
Excited to share CoordTok, a scalable video tokenizer that learns a mapping from coordinates to pixels! With CoordTok, we encode 128-frame videos into only 1k tokens (5 ~ 7x fewer tokens than baselines). Please check our paper for more details!
Paper: arxiv.org/abs/2411.14762
Introducing CoordTok, a scalable video tokenizer that can encode a 128-frame video into only 1k tokens. CoordTok learns a mapping from (x, y, t) coordinates to the corresponding patches of input videos. ๐งต[1/6]
project page: huiwon-jang.github.io/coordtโฆ
Huiwon Jang retweeted
Introducing REPA! We show that learning high-quality representations in diffusion transformers is crucial for boosting generation performance. With REPA, we speed up SiT training by 17.5x (without CFG) and achieve state-of-the-art FID = 1.42 using CFG with the guidance interval. ๐งต[1/7]
sihyun.me/REPA




























