Gaurav

This is silly. But what I find underrated about DeepSeek: they make it common knowledge that Things Work. Things often published years before, by the academia or megacorps; which the community cynically assumed to be BS that doesn't scale. But Whale "gets inspired" to scale them, and in short order these Things, thus validated and derisked, become the backbone of the industry. V2 Fine-Grained MoE with shared experts (Apr/May 2024) – DeepSpeed-MoE (Microsoft Jul 2022); the granuarity of Google's switch-c-2048 (Nov 2022), we still have not exceeded. V3 MTP, Dec 2024 – first from Meta FAIR, Apr 2024 (with a pretty different objective and design tbh). R1 – obviously "inspired by" o1, but based on their internal RLVR project that started with MATH-Shepherd (late 2023) and by Feb 2024 delivered GRPO (itself a tweak of REINFORCE, preceded by RLOO, Kool 2019; now MTS Anthropic btw) in DS-Math. V3.2 DSA (as successor to dense MLA, now used by GLM, in variations by a ton of other Chinese labs and Mistral, soon by Kimi, obviously) – pretty much MQA+ top-k routing, ancient Shazeer as usual, though the indexer was afaict novel. V4 attention sinks and SwiGLU clamping - gpt-oss, Aug 2025 (though they weren't first here, and you can dig deeper, I believed attention sinks will be necessary by May 2024). V4.1 CED, Sep 2026 – YOCO, MSR/Tsinghua May 2024; KV Cache Sharing from ~Shazeer again~ Gemma 3n, Jun 2025 (only Google framed it as an optimization for edge devices like Pixel phones, not datacenter-scale AGI). V4.1 Engram – Google N-Grammer 2022, and Per-Layer Embeddings in that same Gemma 3n and 4 E2B (they even released models). One can continue this list to their smaller side quests like DS-OCR, DS-Prover, whatever. What seems truly indigenous is the priority on low level infrastructure optimization, thus 3FS (inherited from High-Flyer), DeepEP, DualPipe, and much of the rest of their open-sourced stack (obviously building on vLLM with pieces from SGLang and TensorRT-LLM) as well as in-house stuff like DSec and cache management. On the other hand, MLA, auxiliary-loss-free load balancing, and dozens of smaller details (like, just read V4.1 paper) seem to lack direct precursors. TileLang is adopted from Peking University; now all Chinese hardware will use TileLang kernels. DSH, the harness I expect to become the Default Open Source Agent Harness, is at its heart Cordis, adopted from Koishi, a goddamned chatbob with plugins that nobody in the West knew about… My point is that beyond their indigenous creativity, DeepSeek serves a unique role in the ecosystem: they take on the risk to bridge the gap between toy models and papers and the serious frontier. For all the rest to follow them there.