
Ph.D. Student @KAIST_AI AI Startup Founder
Republic of Korea
Joined April 2024
- Tweets188
- Following99
- Followers130
- Likes376
Taewan Cho retweeted
MuJoCo 4 is released!
github.com/google-deepmind/m…
Taewan Cho retweeted
VLAs (from VLMs) ❌ => WAMs (from Video Models) ✅
Why WAMs?
1️⃣ World Physics: VLMs know the internet, but Video Models implicitly model the physical laws essential for manipulation.
2️⃣ The "GPT Direction": VLAs are like BERT (rely heavily on task-specific post-training). WAMs are like GPT (pre-train & prompt), unlocking incredible zero-shot transfer!
What I want to see in 2026:
📈 Scaling Laws: We will see much clearer scaling laws for robotics compared to VLAs.
🤝 Human-to-Robot Transfer: Unlocking massive transfer capabilities using video as a shared representation space.
🤖 Zero-Shot Mastery: Moving from short-horizon tasks to long-horizon, dexterous manipulation without task-specific demonstrations.
We recently open-sourced the checkpoints, training and inference code.
Dive into the research! 👇
📄 Paper: arxiv.org/abs/2602.15922
💻 Code: github.com/dreamzero0/dreamz…
🤗 HF: huggingface.co/GEAR-Dreams/D…
Taewan Cho retweeted
Can we scale up imagination-free WAMs (e.g., Fast-WAM)?
Introducing 𝗗𝗿𝗲𝗮𝗺𝗙𝗿𝗲𝗲, an imagination-free WAM that improves with stronger video models and more video data, running at 11 Hz with a 14B video backbone.
harryjo97.github.io/dreamfre…
🧵[1/13]
LLM 월드모델 JEPA 연구 궁금한사람 이 논문 참고..
이런방향의 훈련연구도 있다. 정도로 생각해보기.
**논문 결과만 보면, 시각-언어 표현을 맞추는 데는 훨씬 적은 학습 샘플이 들고, 표현이 한 모달리티만의 잡음에 덜 치우치는 경향이 있습니다.**
같은 데이터와 109M 인코더를 비교했을 때, good-view JEPA는 next-token prediction이 비슷한 정렬 점수에 도달하는 데 쓴 샘플의 약 **1/42**만 사용했습니다. 다만 이건 *정렬 효율*이지, 언어 모델 학습 전반이 42배 효율적이라는 뜻은 아니에요. [Sample Efficiency](alphaxiv.org/abs/2610.39302?…)
부작용도 **없다고 결론 내릴 수는 없습니다.** 논문이 뒷받침하는 건 좋은 짝을 쓰면 모델이 문장 표현이나 이미지의 촬영 조건 같은 모달리티별 잡음에 덜 매달리고, 장면에 공통된 정보를 더 잘 담는다는 점이에요. 하지만 편향, 사실성, 안전성, 일반적인 능력 저하 같은 부작용은 체계적으로 검증하지 않았습니다. [Nuisance and Alignment](alphaxiv.org/abs/2610.39302?…)
실용적인 학습 문제도 있습니다. 저자들의 실험에서는 설정에 따라 학습이 갑자기 붕괴하는 경우가 있었고, gradient clipping이 이를 줄이는 데 도움이 됐습니다. 따라서 논문이 보여주는 건 **정렬에는 더 효율적인 학습 신호**이지, 모든 종류의 비용과 위험까지 줄어든다는 보장은 아닙니다. [Training Stability](alphaxiv.org/abs/2610.39302?…)
Yann LeCun was right & nobody in this industry wants to say it out loud.
109 million parameters beat 72 billion.
He has spent years saying scale is not the thing that makes a model understand the world. Meta bet the other way. He left in November and started his own lab in Paris.
Last Thursday he published the proof.
The theory everyone has been running on since 2024 is that if you train models big enough, on text, on images, on sound, they all end up with the same picture of reality inside them. Size does it. Nobody ever proved that part.
Randall Balestriero and LeCun proved it. And the proof says size was never doing the work.
What does the work is which two things you show a model together.
A photo and the caption of that photo are two views of one thing. Models trained on pairs like that get there fast.
Two paragraphs pulled from the same document are not two views of one thing. Models trained on those get there slowly and expensively.
Next word prediction is the second kind. That is what every chatbot in the world runs on.
In their tests the first method reached the same understanding after 21.6 million examples. Next word prediction needed 900 million.
And a 109 million parameter text model ended up closer to an image model's view of the world than a 72 billion parameter language model was.
660 times smaller.
Taewan Cho retweeted
Deepseek V4.1 1017 4x RTX PRO 6000のレシピを共有。クラウドと同等の快適さです。
コーディングならPrefill9,000-10,000tok/s, Decode 300-340tok/sくらい出ます。DSpark込の数値なので、ばらつきがあります。
github.com/hikarioyama/deeps…
Taewan Cho retweeted
really excited to see an application built on top of Hydra (bidirectional Mamba arxiv.org/abs/2407.09941)!
at small scales we had done some DNA language modeling and found evidence that Mamba is better than Transformers and convolutions; this paper from a serious bio group shows that it allows modeling at higher nucleotide resolution and improves performance on various genomic prediction tasks
This week we're releasing a preprint exploring state space models for long-sequence genomic prediction, alongside Cerberus, a set of open PyTorch models built on this work. biorxiv.org/content/10.64898…
Taewan Cho retweeted
Speech+RL paper from @Qualcomm AI Research and @KAIST_AI !
For more natural conversational AI, we usually use full-duplex speech language models, which can perform smooth turn-taking and also produce uh-huh-like backchannels.
However, there is no clear answer to when a model should barge in. So in this field, we explore reinforcement learning rather than relying solely on supervised fine-tuning. But due to the nature of full-duplex SLMs, their token distribution is totally different from that of normal language models. So our idea is to build a reinforcement learning method for full-duplex speech language models. Our method, HiPLEX (Hierarchical policy factorization for full-duPLEX SLMs), factorizes the policy into when to talk and what to talk. Plus, our credit assignment method causally attributes rewards to the correct or incorrect timing and semantic decisions. (See Figure 1 featuring the cute San character!)
arxiv: arxiv.org/abs/2610.07727
huggingface paper: huggingface.co/papers/2610.0…
Huge thanks to my Qualcomm speech team.
"Not only does South Korea have the second-highest talent retention rate of 77 percent, its key institution KAIST leapt twelve places since 2022 to now rank as the third-highest institution in terms of producing top AI talent."
So proud of our entire KAIST family—our students, faculty, alumni, and everyone who has helped build this community. Let's keep pushing! 🌟
Full article from Carnegie: carnegieendowment.org/featur…
Taewan Cho retweeted
Pearl (@prlnet) has the fastest FP8 MoE Kernels for Blackwell chips!
Group-Matrix-Multiplication is one of the most optimized operations in AI. As a former researcher at @nvidia I can say firsthand that pushing the performance frontier of MatMuls is... Hard.
Pearl-GEMM is ~5% faster than prior state-of-art Quack (@tri_dao) and ~42% faster than Flashinfer on B200s.
pearlresearch.ai/research/bl…
Introducing GLM-5.3-Flash
- Leading capabilities at a highly competitive price
- Natively multimodal with a 1M-token context window
- A 320B-A18B model released under the MIT License
- Previously previewed as Ox Alpha, running entirely on Chinese AI chips
Blog: z.ai/blog/glm-5.3-flash
Available now across all official platforms:
Weights: huggingface.co/zai-org/GLM-5…
API: docs.z.ai/guides/llm/glm-5.3…
Coding Plan: z.ai/subscribe
ZCode: zcode.z.ai/en
Chat: chat.z.ai
AutoClaw: autoclaw.z.ai
Taewan Cho retweeted
NVIDIA researchers did it again!
They found a way to make KV cache transferable between models.
The target model skips prefill entirely, and the conversion runs 2.7 to 25x faster than processing the context again.
Let's understand why this is so important today.
LLM APIs are stateless, so every turn sends the entire conversation back to the model. The model reads all of it again before writing a single new token, and all of it is billed as input.
Prompt caching allows Anthropic and other providers to hold the KV cache for a stable prefix and bill a hit at roughly 10% of the base input rate, because the compute was already done once.
The 90% reduction is one of the largest lever in LLM serving, which is why so much production work goes into keeping prefixes byte-stable.
But the cache only works on the model that produced it. Keys and values are produced from that model's weights, so no other model can read them.
In pratice, the constraint shows up in LLM routing. If the traffic is shifted to a different model for cost/capability reasons, the accumulated KV cache becomes invalid.
As a result, the accumulated context has to be processed from scratch, and it's billed at full rate.
NVIDIA's recent paper treats this as a representation problem.
Prefill's only output is the KV cache, so to move KV between models, we need to convert one model's cache into the format the other expects.
They first checked whether the conversion has any structure worth exploiting.
They found that moving from Qwen3 14B to 32B, a plain linear regression from a single source layer reconstructed 56% of the variance in the target model's keys.
The two models obviously may have different layer counts, so there is no natural one-to-one pairing between them.
For each target layer they rank every source layer by how well it predicts that layer, then feed the top eight in together, which takes the reconstruction to 79%.
The mapper itself has three parts:
> Each target layer and head gets its own independent linear map, solved in one closed-form step rather than by gradient descent.
> The cross-layer selection described above is the second part, and their ablation shows it carries the most weight of the three.
> Keys also carry a position-dependent rotation from RoPE. They strip that rotation, fit the map in position-free space, then re-apply the target model's rotation at inference.
Across six pairs from Qwen3, Llama 3.1 and Ministral 3, four retain 73 to 98% of the receiving model's standalone accuracy, and the conversion runs 3-25x faster than processing the context again.
Prior work on cross-model KV reuse exists, but it either trains a neural adapter per pair or requires both models to be architecturally identical.
This is probably the first version that is closed-form and training-free, so a lot of it is still open research.
Every pair tested belongs to one family, so it works on Qwen to Qwen and Llama to Llama.
Cross-family transfer is listed as future work.
All six pairs mentioned above also happen to share KV head count and per-head dimension across scales. Mismatched head configurations are currently untested.
The researchers scoped this to dense full-attention only, so sliding-window and attention-recurrent hybrids still need work.
Here's the paper: arxiv.org/abs/2608.03893
Plenty of work is yet to be done. Still, the constraint being solved is genuine.
Every model swap currently invalidates the full KV that was already paid for, and this is the first result showing that work might be recoverable without training anything extra.
That said, all of this only matters because of what the KV cache is doing in the first place.
I wrote a first-principles breakdown of it, covering why the model stores keys and values at all, why the cache grows with every token, and what generation speed looks like with and without it.
Read it below.
If I have 6 RTX Pro 6000s, what would be the best model to run? I want to quickly buy two more to make it 8, though...
Taewan Cho retweeted
Full recipe: DeepSeek-V4-Flash-0731 + DSpark on 2× RTX PRO 6000 Blackwell (SM120)
Several people asked for the exact setup. Everything below is measured on my own system.
## MODEL
Official deepseek-ai/DeepSeek-V4-Flash-0731 (HF)
• FP8, ~167 GB
• Fits in 2×96 GB VRAM as-is
• No requantization required
• The DSpark draft model ships in the same repository
One small tip: downloading individual files with curl was much more reliable than the HF CLI, which repeatedly stalled for us.
---
## ENGINE
Current upstream vLLM and SGLang do not support this model.
You'll need a community vLLM build with:
• SM120 sparse kernels
• DSpark speculative decoding
• Built after Jul 31
The Jul 31 release introduced a new MTP norm layout. Older builds fail during weight loading with:
KeyError: ...mtp_block.main_norm.weight
I'm intentionally not naming the image publicly to avoid sending everyone to the maintainer. Happy to share via DM.
---
## LAUNCH (working configuration)
--tensor-parallel-size 2
--max-model-len 524288
--max-num-seqs 4
--gpu-memory-utilization 0.94
--kv-cache-dtype fp8
--speculative-config '{"method":"dspark","num_speculative_tokens":5}'
--tokenizer-mode deepseek_v4
--tool-call-parser deepseek_v4
--enable-auto-tool-choice
--reasoning-parser deepseek_v4
--block-size 256
--max-num-batched-tokens 2048
--disable-custom-all-reduce
--trust-remote-code
(prefix caching disabled)
num_speculative_tokens must be >= 5 (the DSpark block size). Anything lower is rejected during config validation before CUDA is even initialized.
---
## THE max_num_seqs LIMIT
max_num_seqs > 4 deterministically fails on the first request:
cannot reshape tensor of 0 elements
Root cause:
A hardcoded prefill chunk size (4) feeds an empty padding slice into the sparse MLA kernel.
Confirmed on both community builds we tested.
Cap max_num_seqs at 4.
Additional requests simply queue instead of crashing.
---
## WHAT TO EXPECT
Measured on this setup.
Code generation (temp=0)
• 235-246 tok/s single stream
General prose
• 130-145 tok/s
Aggregate throughput
• c2 ~300 tok/s
• c4 ~400 tok/s
Other numbers
• Prefill: 5-6K tok/s
• 384K needle retrieval: ~80 s
• KV pool: ~988K tokens
Why the spread?
The engine runs at roughly 50 verification steps/s.
Throughput is approximately:
tok/s = 50 × (1 + accepted draft tokens per step)
Acceptance depends heavily on workload:
• Code (temp=0): ~73%
• Prose: ~33%
Don't estimate throughput from SSE delta counts.
Those measure verification steps, not generated tokens.
---
## SANITY LADDER
1. Health check
2. Deterministic text
3. Usage-metered throughput
4. Needle (128K to 384K)
5. Tool calling
6. Confirm the max_num_seqs > 4 signature
If all six pass, you should have essentially the same environment I do.
Questions welcome.
DeepSeek-V4-Flash-0731 + DSpark speculative decoding
2× RTX PRO 6000 Blackwell (TP=2, official FP8, 524K maxlen)
Results (median):
• 243 tok/s single stream — 3.1× over the no-spec baseline
• Aggregate throughput: c2 299 · c4 403 tok/s
• Draft accept: 74–76%, zero failures
I expected an SM120 kernel wall.
Never hit one.
The real bottleneck: max_num_seqs > 4 fails deterministically on the first request. A hardcoded prefill chunk size feeds an empty slice into the sparse kernel.
Capped at 4, it held a 3-hour mixed-load soak at 237–245 tok/s.
Looks like this will be my daily driver for now.
Deepseek-v4-flash for 1 Spark, combining EXL3-3.0bpw & 18.5% prune
Calibrated on Coding, English, Religion, Philosophy, and math. Untested, my Sparks crashed in an auto research run and won't be back on for a day or so
If you have 1 Spark, help me pls.
huggingface.co/0xSero/deepse…
































