Open deep learning compiler stack for CPUs, GPUs and specialized accelerators. Join us for the TVM and Deep Learning Compilation Conference https://nitter.cf/t.co/i6MTbWYt87

Joined January 2018
Apache TVM retweeted
KCoral scales up agentic benchmarking for kernel agents, check it out
Kernel agents can already beat hand-tuned kernels. Spinning up more agents is easy, but scaling GPU evaluation is still hard and expensive. What happens when the agents scale faster than the GPUs? Meet KCoral 🪸: a lightweight shared GPU benchmark environment for agentic kernel development. 🧑‍💻 Agents keep writing and revising code in their own workspace. ⚡ Evaluation is sent to KCoral, which schedules a shared GPU pool and returns correctness and timing. 🌐 Scale across multiple GPUs and nodes through a unified API endpoint. 🔒 Per-request isolation with only milliseconds of execution overhead. By overlapping CPU compilation with GPU work across requests, KCoral further speeds up kernel evaluation. On a single NVIDIA B200, this translates to 2.58× the throughput of sequential local execution.
1
6
22
2,439
Apache TVM retweeted
Great collaboration, and excited to build KCoral together! 🚀 Robust and scalable evaluation is essential for building better kernel agents. Excited to bring what we’ve learned from building PTXBench into KCoral and push this space forward together.
Kernel agents can already beat hand-tuned kernels. Spinning up more agents is easy, but scaling GPU evaluation is still hard and expensive. What happens when the agents scale faster than the GPUs? Meet KCoral 🪸: a lightweight shared GPU benchmark environment for agentic kernel development. 🧑‍💻 Agents keep writing and revising code in their own workspace. ⚡ Evaluation is sent to KCoral, which schedules a shared GPU pool and returns correctness and timing. 🌐 Scale across multiple GPUs and nodes through a unified API endpoint. 🔒 Per-request isolation with only milliseconds of execution overhead. By overlapping CPU compilation with GPU work across requests, KCoral further speeds up kernel evaluation. On a single NVIDIA B200, this translates to 2.58× the throughput of sequential local execution.
4
8
773
Apache TVM retweeted
We build KCoral 🪸 to make GPU access a service for kernel agents. As kernel generation and RL training scale, giving each agent a GPU wastes capacity; leaving agents to coordinate sharing adds work and risks unreliable benchmarks. Agents submit work as needed, and KCoral handles scheduling and isolation—keeping GPUs productive and performance feedback reliable. More in our blogpost 👇
Kernel agents can already beat hand-tuned kernels. Spinning up more agents is easy, but scaling GPU evaluation is still hard and expensive. What happens when the agents scale faster than the GPUs? Meet KCoral 🪸: a lightweight shared GPU benchmark environment for agentic kernel development. 🧑‍💻 Agents keep writing and revising code in their own workspace. ⚡ Evaluation is sent to KCoral, which schedules a shared GPU pool and returns correctness and timing. 🌐 Scale across multiple GPUs and nodes through a unified API endpoint. 🔒 Per-request isolation with only milliseconds of execution overhead. By overlapping CPU compilation with GPU work across requests, KCoral further speeds up kernel evaluation. On a single NVIDIA B200, this translates to 2.58× the throughput of sequential local execution.
6
3
17
2,922
Apache TVM retweeted
Kernel agents can already beat hand-tuned kernels. Spinning up more agents is easy, but scaling GPU evaluation is still hard and expensive. What happens when the agents scale faster than the GPUs? Meet KCoral 🪸: a lightweight shared GPU benchmark environment for agentic kernel development. 🧑‍💻 Agents keep writing and revising code in their own workspace. ⚡ Evaluation is sent to KCoral, which schedules a shared GPU pool and returns correctness and timing. 🌐 Scale across multiple GPUs and nodes through a unified API endpoint. 🔒 Per-request isolation with only milliseconds of execution overhead. By overlapping CPU compilation with GPU work across requests, KCoral further speeds up kernel evaluation. On a single NVIDIA B200, this translates to 2.58× the throughput of sequential local execution.
10
31
10
112
14,268
Apache TVM retweeted
Super excited to co-teach CMU’s ML Systems course with @tqchenml next spring! Looking forward to bringing agentic GPU programming into the classroom! 🚀
We will be teaching Agentic GPU Programming for MLSys as part of the CMU ML Systems course this upcoming spring at @SCSatCMU, touching on knowledge curation, compiler analysis, and feedback loops. We also curated an online book on the topic, check it out mlc.ai/agentic-gpu-programmi…
3
12
145
10,185
Apache TVM retweeted
We will be teaching Agentic GPU Programming for MLSys as part of the CMU ML Systems course this upcoming spring at @SCSatCMU, touching on knowledge curation, compiler analysis, and feedback loops. We also curated an online book on the topic, check it out mlc.ai/agentic-gpu-programmi…
24
164
10
1,069
80,304
Apache TVM retweeted
Excited to share what we’ve been building with TIRx Harness! ⚡ A lot of attention goes to how well agents write kernels. We’re exploring what helps them improve: compiler feedback, a rich kernel zoo, and reliable performance measurements. 🔍
AI agents are becoming GPU kernel engineers. We're rethinking the compiler stack around them. Meet TIRx Harness: an open compiler harness that gives agents an environment to explore, debug, and optimize GPU kernels. 🧱 Minimal stable low-level compiler foundation for predictable GPU programming
🔍 Domain-specific compiler analysis to guide debugging and optimization
📚 Kernel zoo of 60+ kernels and hardware specs to reuse strategies and discover new ones
📊 Remote GPU evaluation for reliable benchmarks and profiling Agent-built Kimi Delta Attention kernels achieved geometric-mean speedups of 2.94x over FlashKDA (forward) and 6.84x over FLA (backward). Our bet: the next leap in agentic GPU programming will come from engineering the environment agents optimize in. Check out our blog: blog.mlc.ai/2026/09/29/tirx-…
3
13
1,428
Apache TVM retweeted
2.94× KDA fwd, 6.84× KDA bwd, 2.59× MSA prefill, 3.99× MSA decode, 1.33× KDA decode, 1.71× MLA, 1.68× VSA — just by pressing Enter. After years of working on ML compilers and designing many abstractions, the most useful compiler thing in TIRx I’ve ever built might just be a PTX tablegen.
AI agents are becoming GPU kernel engineers. We're rethinking the compiler stack around them. Meet TIRx Harness: an open compiler harness that gives agents an environment to explore, debug, and optimize GPU kernels. 🧱 Minimal stable low-level compiler foundation for predictable GPU programming
🔍 Domain-specific compiler analysis to guide debugging and optimization
📚 Kernel zoo of 60+ kernels and hardware specs to reuse strategies and discover new ones
📊 Remote GPU evaluation for reliable benchmarks and profiling Agent-built Kimi Delta Attention kernels achieved geometric-mean speedups of 2.94x over FlashKDA (forward) and 6.84x over FLA (backward). Our bet: the next leap in agentic GPU programming will come from engineering the environment agents optimize in. Check out our blog: blog.mlc.ai/2026/09/29/tirx-…
8
19
7
240
385,064
Apache TVM retweeted
In the age of agentic programming, the compiler's role will increasingly move from just supporting compilation to a flexible harness that agents interact with and rely upon. Compiler harness will be the new frontier in agentic GPU programming.
AI agents are becoming GPU kernel engineers. We're rethinking the compiler stack around them. Meet TIRx Harness: an open compiler harness that gives agents an environment to explore, debug, and optimize GPU kernels. 🧱 Minimal stable low-level compiler foundation for predictable GPU programming
🔍 Domain-specific compiler analysis to guide debugging and optimization
📚 Kernel zoo of 60+ kernels and hardware specs to reuse strategies and discover new ones
📊 Remote GPU evaluation for reliable benchmarks and profiling Agent-built Kimi Delta Attention kernels achieved geometric-mean speedups of 2.94x over FlashKDA (forward) and 6.84x over FLA (backward). Our bet: the next leap in agentic GPU programming will come from engineering the environment agents optimize in. Check out our blog: blog.mlc.ai/2026/09/29/tirx-…
9
26
1
200
37,307
Apache TVM retweeted
AI agents are becoming GPU kernel engineers. We're rethinking the compiler stack around them. Meet TIRx Harness: an open compiler harness that gives agents an environment to explore, debug, and optimize GPU kernels. 🧱 Minimal stable low-level compiler foundation for predictable GPU programming
🔍 Domain-specific compiler analysis to guide debugging and optimization
📚 Kernel zoo of 60+ kernels and hardware specs to reuse strategies and discover new ones
📊 Remote GPU evaluation for reliable benchmarks and profiling Agent-built Kimi Delta Attention kernels achieved geometric-mean speedups of 2.94x over FlashKDA (forward) and 6.84x over FLA (backward). Our bet: the next leap in agentic GPU programming will come from engineering the environment agents optimize in. Check out our blog: blog.mlc.ai/2026/09/29/tirx-…
9
58
10
354
74,301
Apache TVM retweeted
🚀 More reliable agents with DeepSeek V4.1! XGrammar brings strict tool calling to SGLang & vLLM through Structural Tags, enforcing tool argument schemas in DeepSeek’s native format. See how the Structural Tag works 👇 blog.mlc.ai/2026/05/04/xgram… Check out XGrammar 👇 github.com/mlc-ai/xgrammar
4
21
7
89
25,031
Apache TVM retweeted
One thing that I am really excited about this TensorRT release is the first-class support for TVM-FFI, so we can bring custom DSL and agent generated kernels to TensorRT inference nvidia.github.io/TensorRT-Mo…
We just released TensorRT Model Connect in Public Preview. You can take a supported @huggingface model to end-to-end TensorRT inference in just two commands. No intermediate ONNX export, and the resulting bundle can run through native C++ APIs. We also built the entire project with @OpenAIDevs Codex agents, with humans directing and reviewing the work. That includes model implementations, performance tuning, tests, integrations, and docs. It’s open source, so go try it out, dig into the implementations, or contribute support for a new model: github.com/NVIDIA/TensorRT-M…
1
16
1
90
10,190
Apache TVM retweeted
It's cool to learn how different warp/warpgroup roles overlap in the kernel, such as FA4. tvm.apache.org/docs/tirx/nat…
2
11
1
89
6,217
Apache TVM retweeted
We taught a brand-new mini-series this year at @SCSatCMU on Modern GPU Programming for ML Systems, as part of the ML Systems course, touching on fun questions like what data layout swizzling is, how to use 3D TMA, and state-of-the-art Blackwell programming. We released a curated online book based on the materials: mlc.ai/modern-gpu-programmin… check it out
24
290
6
1,827
205,165
Apache TVM retweeted
We release TIRx today, a minimal compiler stack and hardware-native DSL for frontier ML kernels, built around storage-first tensor layouts and reusable tile primitives. tvm.apache.org/2026/06/22/ti… On NVIDIA B200, TIRx delivers up to ~1.08× over cuBLASLt on dense GEMM, outperforms DeepGEMM on all FP8 blockwise workloads with up to ~1.09× speedup, keeps FlashAttention-4 (FA4) typically within ~±2% of CuTeDSL, and remains competitive with cuBLASLt/FlashInfer on NVFP4 GEMM. Through our past experiences building frontier ML kernels, megakernels, and agentic kernel systems, we kept seeing the same boundary problem: new operators and new hardware require new optimization strategies that often break old programming models or compiler passes. TIRx builds on top of Apache TVM and moves toward a simple goal: let users and agents express the best-performing program, even for future hardware generations, while keeping the engineering effort for new kernels and new hardware as low as possible.
4
45
9
151
41,790
Apache TVM retweeted
What will the role of AI compilers be in the age of AI agents and frontier kernel programming? We believe agents should have access to a predictable DSL that offers maximum expressiveness, paired with a minimal compiler they can directly open up, build toolings, and improve for specialized optimizations. TIRx is our effort on this front. We've had a great experience using it in our latest mega-kernel compiler research and teaching Blackwell programming in our ML systems course at CMU. Check it out:
We release TIRx today, a minimal compiler stack and hardware-native DSL for frontier ML kernels, built around storage-first tensor layouts and reusable tile primitives. tvm.apache.org/2026/06/22/ti… On NVIDIA B200, TIRx delivers up to ~1.08× over cuBLASLt on dense GEMM, outperforms DeepGEMM on all FP8 blockwise workloads with up to ~1.09× speedup, keeps FlashAttention-4 (FA4) typically within ~±2% of CuTeDSL, and remains competitive with cuBLASLt/FlashInfer on NVFP4 GEMM. Through our past experiences building frontier ML kernels, megakernels, and agentic kernel systems, we kept seeing the same boundary problem: new operators and new hardware require new optimization strategies that often break old programming models or compiler passes. TIRx builds on top of Apache TVM and moves toward a simple goal: let users and agents express the best-performing program, even for future hardware generations, while keeping the engineering effort for new kernels and new hardware as low as possible.
1
20
127
12,673
Apache TVM retweeted
PithTrain is out. The bigger bet behind it: ML systems built to be evolved by agents, not just maintained by humans. Grateful to my collaborators for everything we've built and learned! Excited for what's next :-)
Two moments every ML researcher knows. You get onto a new cluster, and week one goes to fitting the framework to your setup, not training. A new architecture lands, and trying it means hacking through a gigantic codebase to stay compatible with the pipeline. What you want to change is small. The code you wade through to change isn't. This experience is likely not alone, and many researchers we’ve talked to run into similar issues. A year of this on CMU's FLAME cluster left us with one question: what if a framework were built for an agent to adapt and evolve, not just for humans to maintain? So we introduce PithTrain: a compact, agent-native MoE training system, now ~11K lines of Python, on four principles: - Compact: fits in one context window - Python-native: readable tracebacks, no compiled-extension rebuilds - No implicit indirection: direct calls, each model in its own file - Agent skills: in-repo playbooks for recurring tasks Then we measured the thing nobody measures. Same agent, same tasks, only the framework underneath changes: on PithTrain it finishes with up to 62% fewer turns and 64% less GPU time than production frameworks, while training just as fast. We call this second axis agent-task efficiency, and we believe it deserves to sit alongside training throughput as a metric worth optimizing. Excited to see what people build with it. Built with amazing collaborators @haok1402, Haozhan Tang, Akaash Parthasarathy, @Zichun_Yu. Blog: blog.mlc.ai/2026/06/01/pitht… Code: github.com/mlc-ai/pith-train Paper: arxiv.org/abs/2605.31463
There's a new version of this post
3
14
2,406
Apache TVM retweeted
Two moments every ML researcher knows. You get onto a new cluster, and week one goes to fitting the framework to your setup, not training. A new architecture lands, and trying it means hacking through a gigantic codebase to stay compatible with the pipeline. What you want to change is small. The code you wade through to change isn't. This experience is likely not alone, and many researchers we’ve talked to run into similar issues. A year of this on CMU's FLAME cluster left us with one question: what if a framework were built for an agent to adapt and evolve, not just for humans to maintain? So we introduce PithTrain: a compact, agent-native MoE training system, now ~11K lines of Python, on four principles: - Compact: fits in one context window - Python-native: readable tracebacks, no compiled-extension rebuilds - No implicit indirection: direct calls, each model in its own file - Agent skills: in-repo playbooks for recurring tasks Then we measured the thing nobody measures. Same agent, same tasks, only the framework underneath changes: on PithTrain it finishes with up to 62% fewer turns and 64% less GPU time than production frameworks, while training just as fast. We call this second axis agent-task efficiency, and we believe it deserves to sit alongside training throughput as a metric worth optimizing. Excited to see what people build with it. Built with amazing collaborators @haok1402, Haozhan Tang, Akaash Parthasarathy, @Zichun_Yu, @junrushao, Todd Mowry, @XiongChenyan and @tqchenml. Blog: blog.mlc.ai/2026/06/01/pitht… Code: github.com/mlc-ai/pith-train Paper: arxiv.org/abs/2605.31463
3
39
7
175
24,529
Apache TVM retweeted
Excited to share XGrammar-2 has been accepted by #ACMCAIS 2026 @CAISconf! 🎉 ⚡️ Up to 80x speedup 🛡️ Strict tool-calling correctness 🚀 Trusted by xAI in production systems Join our presentation today at 4 PM at Bayshore Ballroom to see how we power critical agent workloads at scale. 🔗Blog: blog.mlc.ai/2026/05/04/xgram… 🔗GitHub: github.com/mlc-ai/xgrammar
Introducing XGrammar-2: structured generation for complex agent harnesses. Strict tool-calling formats. Built-in DeepSeek-V4 and Qwen-3.6 support. Up to 80x speedup over XGrammar. Ready-to-use integrations with vLLM, SGLang, TensorRT-LLM, and more! ⚡ From Claude Code to OpenClaw, agents are defining more complex harnesses. XGrammar-2 ensures LLMs always interact with them in the right way. Built in collaboration with DeepSeek, Databricks, and leading frontier AI labs to bring XGrammar-2 into latest models and products. 🧩 Structural Tag: one unified abstraction to describe any format your agent needs 🚀 Scales to 500+ strictly typed tools for complex agent harnesses 🌐 Native APIs in Python, C++, Rust, and JS, running everywhere from cloud to edge 🛠️ Integrated with vLLM, SGLang, TensorRT-LLM, and more Excited to see what agent builders create with it! Blog: blog.mlc.ai/2026/05/04/xgram… GitHub: github.com/mlc-ai/xgrammar
1
12
40
4,435
Apache TVM retweeted
Had a blast presenting Event Tensor at MLSys2026! 🚀 TL;DR: We make writing dynamic megakernels simple. ⚡ Thanks to my advisors & collaborators. Here’s pic from yesterday’s poster session 👇 Let’s chat! 🔗 arxiv.org/abs/2604.13327
2
13
3
54
19,905