Another step forward. Congrats to the team!
Day-0 support for @Kimi_Moonshot K3.1 Pro in TokenSpeed.
Sparse attention for long contexts. With KV sharding and decode context parallelism, KV cache capacity scales from 2.43M to 16.35M tokens on 8× NVIDIA GB300.
Available on AMD MI350X, @NVIDIAAI Blackwell, and NVIDIA Vera Rubin.
📖 lightseek.org/blog/tokenspee…
⭐️ github.com/lightseekorg/toke…
Third post in our AMD GPU engineering deep-dive series!
Jungwook walks through warp-pipelining (aka block ping-pong) across MI300, MI350, and MI450: explicitly staggering wave groups to overlap matrix compute with data preparation, achieving performance on par with hand-tuned hipBLASLt assembly for compute-bound GEMMs.
Another showcase of the team's deep investment in foundational optimization techniques: exposing low-level scheduling control through high-level Gluon block programming. Less code, fewer tokens (especially for agents!), easier to verify, and no need to trade away performance.
rocm.blogs.amd.com/software-…
Lei Zhang retweeted
EAGLE-4 is here—a game changer for agentic coding, with massive decoding speedups.
Honored to be a day-0 engine partner with @EagleCorp
Train with TorchSpec. Serve with TokenSpeed. Training and inference in one diagram 👇
Technical deep dive: lightseek.org/blog/eagle-4
Lei Zhang retweeted
TokenSpeed Kernel at today’s Triton Conference poster session!
Portable APIs and high-performance kernels for multi-silicon LLM inference. The AMD backend is fully rewritten in Triton + Gluon for ease of use and speed. 👇
⭐️ github.com/lightseekorg/toke…
At Triton Developer Conference today! My team is presenting three talks/posters covering AMD MI455X performance, portable+performant LLM inference kernels, and rack-scale communication on AMD Helios:
* Out-of-the-Box Performance on AMD MI450 with Triton Gluon
* Inside TokenSpeed-Kernel: Portable, High-Performance LLM Inference Kernels Across GPUs
* Programming the Rack from Triton: Symmetric Communication on AMD Helios – MI455X
Come check them out and chat with us! :)
TokenSpeed is now SOTA for Kimi K3 on AMD: 1.4× faster than ATOM. 🚀
How TokenSpeed got here:
🔹 AMD is a first-class citizen, not an afterthought
🔹 Every AMD kernel rewritten in Triton + Gluon
🔹 Built with the amazing AMD Triton team 🐐
Just getting started. Much more coming. Technical details 👇
amd.com/en/developer/resourc…
Kimi K3 on AMD Instinct MI355X, powered by TokenSpeed ⚡
216 output tok/s per user on 8× MI355X: 1.4× the decode speed of ATOM at C1, while running higher precision.
Measured on a real agentic workload: multi-turn SWE-smith, 50K+ context, 10–15 turns.
Technical deep dive with @AIatAMD 🧵
Lei Zhang retweeted
Triton first for portability, then agent-assisted Gluon kernels where profiling says it matters. Both follow the same numerical contracts.
Thanks @LeiLMx and the AMD Triton team 🐐
📖 amd.com/en/developer/resourc…
⭐ github.com/lightseekorg/toke…
Lei Zhang retweeted
Kimi K3 on AMD Instinct MI355X, powered by TokenSpeed ⚡
216 output tok/s per user on 8× MI355X: 1.4× the decode speed of ATOM at C1, while running higher precision.
Measured on a real agentic workload: multi-turn SWE-smith, 50K+ context, 10–15 turns.
Technical deep dive with @AIatAMD 🧵
Great work by the team advancing state-of-the-art Kimi K3 inference on 8× AMD MI355X GPUs with TokenSpeed: 216 output tok/s per user at C1, 1.4× ATOM's performance, while using higher precision!
Our new technical deep dive covers optimizations across KDA, MLA, AttnRes, LatentMoE, and multi-GPU communication—all built on a portable Triton foundation, followed by agent-assisted Gluon specialization.
amd.com/en/developer/resourc…
What a rewarding journey! I still remember all those days working together with OpenAI on MI300, MI350, and MI450 PoCs—pushing compiler support forward and squeezing out kernel performance.
It’s amazing to see the team’s great work pay off at this scale. Onto the next chapter: making Triton and Gluon absolutely hum on these beasts. 🚀
Look who showed up as another AMD Helios system enters the wild! 👋 @OpenAI
Last week, Vamsi Boppana and @udayruddarraju joined the team for a lab tour as we continue pushing the frontier of AI infrastructure, together.
+1. It's a deliberate design choice in TokenSpeed: every kernel is required to have a baseline portable Triton impl (pytorch.org/blog/lightseek-t…).
The immediate benefit is portability—a new platform can work from day one with minimal efforts. But increasingly, the more important role is as a semantic contract: a simple implementation that precisely captures the tensor semantics and serves as the numerical reference for more aggressively specialized kernels.
From there, agents can explore Gluon impl with much more explicit, low-level control over scheduling, memory movement, layouts, etc.
Languages as contracts and compilers as verifiers. It's also why @cdleary's nitter.cf/cdleary/article/209487… resonates much. The language abstraction is no longer only about shielding humans from hardware complexity; it also defines the search boundary within which agents can optimize safely.
And that contract remains valuable for humans too: something understandable to reason, debug, and maintain—even when the optimized impl becomes increasingly machine-generated. :)
haha I thought you wouldn't notice and it will fly by 😀 I wrote a version of github.com/francois141/ptx_g… targeting AMD GPUs and will let it "hill climb" to see if we can automate the perf. Helps that all a the tokenspeed kernels are triton for the base kernels.
Visualizing to learn model internals in the agentic era can be surprisingly low-friction.
I had Codex one-shot architecture notes earlier for several recent OSS models—DeepSeek V4/V4.1, GLM 5.3, and Kimi K3—specifically from a kernel-development perspective: tensor contracts, exact shapes/layouts, attention variants, KV-cache representation, precision choices, dataflow, and operator boundaries.
Sharing in case they’re useful to others:
lei.chat/models/
Second episode of our Triton/Gluon MI455X engineering series: a deep dive into shared memory (LDS) optimization.
It talks about transposed LDS loads, plus LDS partition conflicts, a new consideration on CDNA5, and how we design suitable layouts and allocation to avoid them elegantly.
Lots of low-level hardware, compiler, and layout details worth a read:
rocm.blogs.amd.com/software-…
Triton/Gluon AMD mi455x/cdna5 API documentation available at triton-lang.org/main/gluon/a…, together with earlier generations.
Lei Zhang retweeted
🎉 Congrats to @Zai_org on GLM-5.3-Flash — the first natively multimodal model in the GLM-5 series, and their first model combining sparse and linear attention.
GLM-5.3-Flash introduces a new combination of linear attention for local dependencies and sparse attention for global context, with IndexPool keeping the indexing overhead low even at 1M-token context.
This new architecture also brings a new set of inference challenges. TokenSpeed already has day-0 support for GLM-5.3-Flash, with the stack validated on both @NVIDIA and @AMD GPUs. 👇
lightseek.org/tokenspeed/rec…
Introducing GLM-5.3-Flash
- Leading capabilities at a highly competitive price
- Natively multimodal with a 1M-token context window
- A 320B-A18B model released under the MIT License
- Previously previewed as Ox Alpha, running entirely on Chinese AI chips
Blog: z.ai/blog/glm-5.3-flash
Available now across all official platforms:
Weights: huggingface.co/zai-org/GLM-5…
API: docs.z.ai/guides/llm/glm-5.3…
Coding Plan: z.ai/subscribe
ZCode: zcode.z.ai/en
Chat: chat.z.ai
AutoClaw: autoclaw.z.ai
Nice to see another major milestone! It has been a great journey to work with the incredible community to explore, learn, iterate, and contribute.
Around GTC, we started thinking about building a new LLM inference engine. Some told us: “it’s too late in 2026.”
We believed there was still room for a different approach.
Starting in mid-March, we designed and built TokenSpeed from first principles, focusing on correctness, performance, and a modern architecture. We open-sourced TokenSpeed in early May, with scheduler and kernel abstractions that take a different path toward simplicity, extensibility, and a multi-silicon future.
In just a few months, TokenSpeed has collaborated with NVIDIA, AMD, Triton, LLVM, Qwen, Mooncake, and FluentLLM communities, supported Inkling and Kimi K3 at Day 0, and joined the PyTorch Ecosystem.
Everything is still early. New hardware, models, and workloads are reshaping LLM inference.
We’ll keep building and moving fast.
Lei Zhang retweeted
Power of OPEN: Mi455/CDNA5 ISA: amd.com/content/dam/amd/en/d…
MI450X launched last week. We’re kicking off an engineering blog series on its hardware features and how to use them from Gluon.
First one is from @borontion and team on decode attention: using explicit WMMA layouts, async TDM→LDS loads, four-stage triple-buffered pipelining, and fused Split-K to deliver 17.1 TB/s, 85% of peak HBM.
rocm.blogs.amd.com/software-…
Great work from team to enable Kimi K3 on AMD GPUs at day 0 within just one week and push on various kernel optimizations!
Kimi K3 is now supported in TokenSpeed.
We worked closely with the @Kimi_Moonshot team to bring Day 0 support to both NVIDIA Blackwell ((G)B200/(G)B300) and AMD Instinct (MI350X/MI355X) in just one week.
✅ Prefix caching
✅ Speculative decoding
✅ Disaggregated serving
✅ CUDA Graph decode
We also share how we optimized KDA, Gated MLA, Stable LatentMoE, AttnRes, and our unified Flat KV architecture.
Blog ↓
lightseek.org/blog/tokenspee…
ROCm 7.14 is here.
Meet TheRock, the new production-ready open-source build and release system for ROCm, alongside expanded AI hardware support, updated frameworks, stronger developer tools, and simplified deployment.
See what's new and start building: rocm.blogs.amd.com/ecosystem…


















