Robots still need teleoperated demos for each new task.
Human videos of the task are cheap, but visual and action gaps make them hard for robots to learn from.
👉 RoboTryOn uses virtual try-on to turn human videos into robot demonstrations.
🌐 ryan-seungyong-lee.github.io…
🧵 [1/8]
Human videos are also cheap: matching teleop-only training takes about 1/3 of the collection cost.
📄 arxiv.org/abs/2611.05163
💻 github.com/ryan-seungyong-le…
🤗 huggingface.co/RyanL22/RoboT…
🕹️ huggingface.co/spaces/RyanL2…
Joint work with @gloryhyeok24, Minyoung Kim, and Jinwoo Shin.
[8/8]
34
Beyond replay, IDM labels match the teleoperated action distribution; retargeted ones do not.
Our data yields policy gradients aligned with those of teleop data (cosine 0.53 vs. ≤0.17), and the trained policy stays in the teleop action range on new tasks (55% vs. ≤10%).
[7/8]
26
Action gap: instead of retargeting the human hand, we label the synthesized robot video with an inverse dynamics model.
Replayed open loop in simulation, our labels succeed 2× as often (69.7% vs. 34.8%): retargeted hands reach the object but fail to grasp it.
[6/8]
22
Visual gap: our synthesized frames overlap real robot frames in the policy's feature space (Fréchet SigLIP distance 125.5 → 32.7).
So, unlike masking (EgoMimic) or overlay (Masquerade), RoboTryOn needs no post-processing at test time.
[5/8]
28
From human videos alone, the policy learns tasks with no teleoperated demonstration: new objects, goals, skills, and even bimanual tasks, though every teleoperated demo is single-arm.
Success on these 7 tasks: 2% → 60% (π0.5, avg. over OpenArm and RB-Y1).
[4/8]
30
How it works:
1️⃣ An image model translates human keyframes into the target robot.
2️⃣ A video model fills in the motion, guided by the hand skeleton.
3️⃣ An inverse dynamics model labels the joint actions.
[3/8]
33
Virtual try-on keeps the person and swaps the garment.
RoboTryOn keeps the demonstrated motion and swaps the human for the robot, while preserving the task, objects, and background.
One set of models serves different embodiments.
80
Seungyong Lee (Ryan) retweeted
A recent line of WAMs generates actions from VDM hidden states of a single denoising step, yet their quality gets little attention.
👉 We introduce FRED to improve them via self-distillation, with large gains on real robot experiments.
🌐 huiwon-jang.github.io/FRED/
🧵 [1/7]
Seungyong Lee (Ryan) retweeted
We've been cooking up something BIG 🌋
Now bridging to @LineaBuild via Across can earn you 50 LXP!! The Linea OG Rush campaign is live from May 22 - June 4 on @IntractCampaign 💫
Earn your LXP here: link.intract.io/LineaOGRush
Seungyong Lee (Ryan) retweeted
Let's see who wins in a race to @LineaBuild Across, Orbiter, Stargate or Hop?
Seungyong Lee (Ryan) retweeted
🔥 $MODE Airdrop! #Bybit x #MODE: 14,000,000 $MODE Up for Grabs!
To do:
1. Follow @Bybit_Official and @modenetwork
2. Quote/Re-tweet this post.
3. Follow the steps in the announcement link to win
💫 Join Event: i.bybit.com/1pBosabq
#TheCryptoArk #BybitTrading
Seungyong Lee (Ryan) retweeted
Season 2 starts today! (5AM UTC)
Episode 01 has kicked off!
Grab your referral codes from Season 1 participants and let's start Season 2 together!
More details:
link.medium.com/4eM7qf7gPGb
🏠Airdrop page: airdrop.gomble.io
📖S2 Ep01: ~ Feb. 14, 2024 (4 AM UTC)
Seungyong Lee (Ryan) retweeted
A new milestone for GOMBLE! 🎊
Today we release our Vision Paper!
What to find out more about Game Ecosystem, Tokenomics, and more?
📜Vision Paper → gomblegames.gitbook.io/visio…
📄Medium → medium.com/@gomblegames/gomb…
Seungyong Lee (Ryan) retweeted
Alpha from today’s Town Hall meeting 👀
We are proud to reveal our long-awaited token ticker and image.
🪙 Official ticker: $G
🗣️ Official name: to be announced
Seungyong Lee (Ryan) retweeted
Wen NFT? Wen DAO?
SpaceKids👽 of the $G Star have an answer!
READ THIS👇
📝medium.com/@gomblegames/spac…
⚠️Beware of the scams, we will never do surprise mints.
Seungyong Lee (Ryan) retweeted
STAKE or DARE?
Finally, put your MM to work!
📅Open: 2024-02-29, 05:00AM UTC
Wondering how to play with MM and get NFTs?
Find out NOW!👇
medium.com/@gomblegames/stak…
Seungyong Lee (Ryan) retweeted
GOMBLE raised $10M funding in total.
We firmly believe in the power of collaboration and community in game development.
GOMBLE has secured a total of $10 million in cumulative funding, with recent participation from notable global venture capital firms such as @TheSpartanGroup @hashed_official @IOSGVC @ForesightVen @BigBrainVC and many more. This adds to the initial round from 2023, which had participation from @BinanceLabs @animocabrands @altosvc @shimacapital and others.
Sorry, there were so many that we couldn’t tag them all…😅
Ready to go even higher! Let’s $G! 🚀
More details -> medium.com/@gomblegames/gomb…
Seungyong Lee (Ryan) retweeted
OG Spaceship🚀 is almost ready to launch!
$G tokens and right to mint 5 SpaceKids👽 NFT
Find out what more you can gain with OG Spaceship🚀
medium.com/@gomblegames/08b9…
Seungyong Lee (Ryan) retweeted
Test Auction is going live!
Bid to win!
If you win - you win a special Test Auction Winner NFT badge.
For all participants, more bids – better rewards.
Bid here 👉 auction.gomble.io/













