Huiwon Jang (@huiwon0516) profile picture

PhD Student @kaist_ai | Research @RLWRLD_ai

Joined April 2023
Excited to introduce FRED, a WAM framework improving zero-shot generalization and human-to-robot transfer! ๐Ÿค– We view VDM as future representation generators and improve them via self-distillation and video conditioning. ๐ŸŒ huiwon-jang.github.io/fred ๐Ÿค— huggingface.co/FRED-AGI-LAB
A recent line of WAMs generates actions from VDM hidden states of a single denoising step, yet their quality gets little attention. ๐Ÿ‘‰ We introduce FRED to improve them via self-distillation, with large gains on real robot experiments. ๐ŸŒ huiwon-jang.github.io/FRED/ ๐Ÿงต [1/7]
7
212
Huiwon Jang retweeted
๐Ÿค– Which robot policy checkpoint should you deploy? Lower validation loss can point you to the wrong one. Introducing ๐—ฅ๐—”๐—–๐—˜: ๐—–๐—ฎ๐—น๐—ถ๐—ฏ๐—ฟ๐—ฎ๐˜๐—ฒ๐—ฑ ๐—–๐˜‚๐—บ๐˜‚๐—น๐—ฎ๐˜๐—ถ๐˜ƒ๐—ฒ ๐—ฆ๐—ฐ๐—ผ๐—ฟ๐—ถ๐—ป๐—ด for rollout-free checkpoint selection. RACE ranks checkpoints from a fixed set of successful reference trajectories, without running each candidate on the robot. ๐Ÿงช Simulation: stays reliable when references shift from demos to noisy rollouts, while validation loss turns negative ๐Ÿฆพ Real robots: 1,800 evaluations on 6 tasks. RACE is highly correlated with real deployment success rate for both Diffusion Policy and ฯ€0.5-DROID, while baselines failed. ๐ŸŽฌ 60-second overview below ๐ŸŒ changyeon.site/race/ ๐Ÿงต 1/8
3
23
57
2,036
Huiwon Jang retweeted
Robots have learned from robot demos. But can they learn new tasks from human experience alone? ๐ŸŽค CoRL 2026 Scaling H2R Workshop - Oral Presentation Introducing H2RB: Human-to-Robot Bridge. We bridge perception and action gaps to transfer new skills and scale human experience across robot embodiments. ๐Ÿงต ๐Ÿ“„ Paper: arxiv.org/abs/2611.04391 ๐ŸŒ Project: junhyeong-park.me/h2rb/ ๐Ÿค— HF Paper: huggingface.co/papers/2611.0โ€ฆ ๐Ÿ–ฅ๏ธ Code: github.com/jun-brro/h2rb-codโ€ฆ
5
10
38
1,817
Huiwon Jang retweeted
๐Ÿค– Improving robot foundation models with real-world RL, then continually carrying that experience forward across tasks. Introducing ๐—”๐—–๐—–๐—ฅ๐—จ๐—˜: ๐—ฐ๐—ผ๐—ป๐˜๐—ถ๐—ป๐˜‚๐—ฎ๐—น ๐—ผ๐—ณ๐—ณ-๐—ฝ๐—ผ๐—น๐—ถ๐—ฐ๐˜† ๐—ฅ๐—Ÿ for robot foundation models. Our real-world policies learn through physical robot interaction, starting from a small set of demos. We inherit the actorโ€“critic across tasks and distill earlier behavior, so experience accumulates while prior skills remain. ๐Ÿ’ป ๐—Ÿ๐—ผ๐˜„ ๐—ฎ๐—ฑ๐—ฑ๐—ถ๐˜๐—ถ๐—ผ๐—ป๐—ฎ๐—น ๐—š๐—ฃ๐—จ ๐˜๐—ฟ๐—ฎ๐—ถ๐—ป๐—ถ๐—ป๐—ด ๐—ผ๐˜ƒ๐—ฒ๐—ฟ๐—ต๐—ฒ๐—ฎ๐—ฑ: RL updates a compact actorโ€“critic while the foundation model stays frozen. โšก Subsequent tasks reach 95% success in about ๐Ÿฐโ€“๐Ÿญ๐Ÿฎ ๐—บ๐—ถ๐—ป๐˜‚๐˜๐—ฒ๐˜€ ๐—ผ๐—ณ ๐—ฟ๐—ผ๐—ฏ๐—ผ๐˜ ๐—ถ๐—ป๐˜๐—ฒ๐—ฟ๐—ฎ๐—ฐ๐˜๐—ถ๐—ผ๐—ป. On real robots: ๐Ÿ“‰ Up to 68.5% less robot interaction vs. single-task RL ๐ŸŽฏ 98.3% final prior-task success ๐ŸŽฌ A 60-second overview below. ๐ŸŒ accrue-rl.github.io/ ๐Ÿงต 1/6
3
33
3
141
7,201
Introducing SQAM, Q-Learning with Scalar Adjoint Matching! ๐Ÿš€ Off-policy RL for flow policies, without a single VJP through the policy. Bigger gains on the hardest tasks, at a lower cost per update. ๐Ÿงต [1/8]
7
16
55
8,327
๐Ÿ†1st place in the AI Scientist track at Ralphthon @ ICML2026 โ€” $10,000 in OpenAI credits. @happyhappy_jun & @WoominSong from @KAIST_AI We've open-sourced all of it: harness, code, the full log of research traces. See our post: byungjunyoon.ai/writing-drivโ€ฆ How we did it ๐Ÿงต
6
18
41
5,006
Huiwon Jang retweeted
๐ŸŽผ Can we turn music into readable code? Excited to share Decomposer, a post-training framework for symbolic music decompilation! Given MIDI, Decomposer recovers an executable program in Strudel that reconstructs the music while exposing patterns, harmony, rhythm, and voices as editable code. ๐Ÿงต [1/5]
11
48
8
276
45,616
Huiwon Jang retweeted
Can MLLMs actually track what's happening in a video? Introducing VSTAT ๐ŸŽฏ, our new benchmark for visual state tracking. The tasks are simple: count cups, read typed words, count page flips. Humans solve them easily. MLLMs don't. vision-x-nyu.github.io/vstatโ€ฆ ๐Ÿงต [1/11]
11
65
7
242
170,941
Introducing TRQAM! Internalizing a KL trust region inside the sampling SDE stabilizes off-policy RL fine-tuning of pretrained flow policies. With TRQAM, we lift offline RL success on 50 OGBench tasks from 46% to 68%. ๐Ÿงต [1/8] yonghdong.github.io/blog/trqโ€ฆ
8
22
1
50
5,038
Huiwon Jang retweeted
RLDX-1 took the first place "officially" in the RoboCasa365 benchmark! robocasa.ai/leaderboard.html Tech report: arxiv.org/abs/2605.03269
1
29
3
58
15,908
Excited to share RLDX-1, a robot foundation model for human-like dexterous manipulation in the real world! It integrates versatile intelligence with functional capabilities, outperforming ฯ€โ‚€.โ‚… and GR00T N1.6 across simulation and real-robot tasks. ๐ŸŒrlwrld.ai/rldx-1
(1/12) Hi, we are #RLWRLD (ReaL WoRLD). RLDX-1 is live. Dexterity is intelligence, and it lives in #RLDX (RealDex). A dexterity-first foundation model from for robot hands that builds muscle memory through motion, history, and contact. - Across 10 dexterous real-world tasks on the ALLEX humanoid and DROID setup, RLDX-1 ~2ร— outperforms ฯ€โ‚€.โ‚… and GR00T N1.6. - SOTA on 8 simulation benchmarks in LIBERO, SIMPLER, and RoboCasa. Everything ships today: training and inference code, the pre-trained model, mid-trained checkpoints, and a fine-tuned checkpoint for every reported benchmark. LoRA recipes for parameter-efficient adaptation to your robot. Supports LeRobot v2.1 datasets. A one-line CLI to toggle motion / memory / physics modules per embodiment. Full technical report with ablations. Not just our cool demos. We document the architecture, data, and design decisions behind every result. Bring your own robot. Train and deploy RLDX-1 on it. ๐Ÿงต What's inside RLDX-1, in 12 posts. ๐ŸŒ rlwrld.ai/rldx-1 ๐Ÿ“„ arxiv.org/abs/2605.03269 ๐Ÿ’ป github.com/RLWRLD/RLDX-1 ๐Ÿค— huggingface.co/collections/Rโ€ฆ
1
7
331
Huiwon Jang retweeted
RLDX-1: A Vision-Language-Action model for human-like dexterous manipulation Built on the Multi-Stream Action Transformer, it unifies heterogeneous modalities with motion awareness, long-term memory, and physical sensingโ€”achieving 86.8% on ALLEX tasks versus ~40% for ฯ€0.5 and GR00T N1.6.
3
23
78
5,449
Huiwon Jang retweeted
A Design Space for Live Music Agents ๐ŸŽท๐ŸŽน๐Ÿฅ #CHI2026 What does it take for AI to truly jam with you? We surveyed 184 live music agents across AI, HCI, and Computer Music fields to map the design space, and where it's headed. ๐Ÿ—“๏ธ Talk: Fri Apr 17, 12:15PM ยท P1 Room 132 ๐Ÿ“„ Paper: arxiv.org/abs/2602.05064 ๐Ÿ”— Interactive demo: live-music-agents.github.io/
1
14
48
3,420
Huiwon Jang retweeted
Catastrophic forgetting has long been a challenge in continual learning. However, our new study found that pretrained Vision-Language-Action (VLA) models are surprisingly resistant to forgetting! Zero forgetting, or even positive backward transfer, is possible with simple experience replay. arxiv.org/abs/2603.03818
19
108
15
666
61,477
Excited to share Adaptive Low-Pass Guidance (ALG): a simple training-free, drop-in fix that brings dynamic motion back to Image-to-Video models! Demo videos, paper, & code below! choi403.github.io/ALG (๐Ÿงต 1/7)
3
38
4
185
17,371
Huiwon Jang retweeted
Meet MELD: a masked diffusion model (MDMs) designed for de novo molecule generation. MELD assigns per-element learnable noise schedule that tailors noise at the atom & bond level to avoid state-clashing problem. With MELD we achieve state-of-the-art property alignment in multi-conditioned generation and a >70% boost in chemical validity. Dive in ๐Ÿ‘‰ holymollyhao.github.io/MELD/ [1/7]
1
6
1
26
3,379
Excited to present FastTD3: a simple, fast, and capable off-policy RL algorithm for humanoid control -- with an open-source code to run your own humanoid RL experiments in no time! Thread below ๐Ÿงต
15
113
14
558
133,023
Huiwon Jang retweeted
Exicited to share my internship project ๐Ÿ˜„ Great experience to work with awesome people. If you are interested in improving LLM pretraining, concepts, and distillation, please take a look!
๐Ÿฅฅ๐ŸŒช๏ธ Introducing CoCoMix - a LLM pretraining framework that predicts concepts and mixes them into its hidden state to improve next token prediction. ๐Ÿ“ˆ More sample-efficient and outperforms next token prediction, knowledge distillation, and inserting pause tokens. ๐Ÿ”ฌBoosts interpretability and steerability by analyzing and modifying the predicted concepts. ๐Ÿ“: arxiv.org/abs/2502.08524
6
18
131
13,418
Excited to share CoordTok, a scalable video tokenizer that learns a mapping from coordinates to pixels! With CoordTok, we encode 128-frame videos into only 1k tokens (5 ~ 7x fewer tokens than baselines). Please check our paper for more details! Paper: arxiv.org/abs/2411.14762
Introducing CoordTok, a scalable video tokenizer that can encode a 128-frame video into only 1k tokens. CoordTok learns a mapping from (x, y, t) coordinates to the corresponding patches of input videos. ๐Ÿงต[1/6] project page: huiwon-jang.github.io/coordtโ€ฆ
2
9
975
Huiwon Jang retweeted
Introducing REPA! We show that learning high-quality representations in diffusion transformers is crucial for boosting generation performance. With REPA, we speed up SiT training by 17.5x (without CFG) and achieve state-of-the-art FID = 1.42 using CFG with the guidance interval. ๐Ÿงต[1/7] sihyun.me/REPA
6
46
3
289
30,476