cofounder @wafer_ai ⿻ prev @twosigma, @google, @seinetwork, @uchicago
sf
Joined May 2025
- Tweets4.8K
- Following1.5K
- Followers6.3K
- Likes4K
wowza!
Took Wafer 4 months to get to 1 trillion tokens a day on @OpenRouter, but only 4 days to get to 2 trillion!
steve retweeted
Took Wafer 4 months to get to 1 trillion tokens a day on @OpenRouter, but only 4 days to get to 2 trillion!
steve retweeted
listen to @gpusteve. he knows a thing or two about ai/gpu perf.
four nvidia resources covering:
- how cuda device code becomes ptx and cubins, how fatbins package gpu code, and when the driver compiles ptx at runtime.
- gpu targets, device linking, intermediate files, and compiler reports for registers, shared memory, and spills.
- ptx state spaces, memory ordering, synchronization, matrix instructions, and accumulation precision.
- reading generated sass and checking register counts and static memory usage with cuobjdump, plus inspecting register liveness with nvdisasm.
useful when your "optimized" kernel gets slower
if i had to learn cuda compilation, ptx, register spills, and gpu disassembly from scratch because my "optimized" kernel got slower, i'd start here
links in thread 🧵
the compilation and machine code section of our ai performance engineering repo contains four nvidia resources for understanding how cuda source becomes gpu instructions and inspecting the code behind ai kernels:
understanding ptx explains how nvcc separates cpu and gpu code, why ptx is a virtual instruction set, and how ptxas turns it into a cubin.
the article covers fatbins, driver jit compilation, and gpu compatibility, giving context for deployment targets and inference startup costs.
nvcc compiler driver covers gpu architecture targets, the files each compilation stage produces, separate compilation, and device linking.
the nvcc guide explains how to retain intermediate files and report register usage, shared memory, and spill loads and stores. those compiler reports help compare kernel builds after changes to tile size, unrolling, or fusion.
ptx isa defines register, local, shared, and global state spaces; memory ordering and synchronization; and arithmetic and matrix instructions.
ptx isa details operand layouts, data types, accumulation precision, and gpu requirements, plus completion rules for asynchronous copies. useful for checking data movement and numerical behavior in attention and matmul kernels.
cuda binary utilities covers extracting embedded ptx and cubins, disassembling native gpu instructions (sass), and reporting register counts and static memory usage with cuobjdump.
nvdisasm provides control-flow graphs when the cubin contains complete relocation information, plus register-liveness views.
with debug line information, nvdisasm can annotate instructions with source lines. useful for inspecting generated loads, stores, branches, and arithmetic after a kernel rewrite.
together, these resources connect the compilation process to the code being measured. they help identify what changed between kernel builds and what to investigate before the next correctness check or benchmark.
follow and save to keep up with the series.
if i had to learn cuda compilation, ptx, register spills, and gpu disassembly from scratch because my "optimized" kernel got slower, i'd start here
links in thread 🧵
the compilation and machine code section of our ai performance engineering repo contains four nvidia resources for understanding how cuda source becomes gpu instructions and inspecting the code behind ai kernels:
understanding ptx explains how nvcc separates cpu and gpu code, why ptx is a virtual instruction set, and how ptxas turns it into a cubin.
the article covers fatbins, driver jit compilation, and gpu compatibility, giving context for deployment targets and inference startup costs.
nvcc compiler driver covers gpu architecture targets, the files each compilation stage produces, separate compilation, and device linking.
the nvcc guide explains how to retain intermediate files and report register usage, shared memory, and spill loads and stores. those compiler reports help compare kernel builds after changes to tile size, unrolling, or fusion.
ptx isa defines register, local, shared, and global state spaces; memory ordering and synchronization; and arithmetic and matrix instructions.
ptx isa details operand layouts, data types, accumulation precision, and gpu requirements, plus completion rules for asynchronous copies. useful for checking data movement and numerical behavior in attention and matmul kernels.
cuda binary utilities covers extracting embedded ptx and cubins, disassembling native gpu instructions (sass), and reporting register counts and static memory usage with cuobjdump.
nvdisasm provides control-flow graphs when the cubin contains complete relocation information, plus register-liveness views.
with debug line information, nvdisasm can annotate instructions with source lines. useful for inspecting generated loads, stores, branches, and arithmetic after a kernel rewrite.
together, these resources connect the compilation process to the code being measured. they help identify what changed between kernel builds and what to investigate before the next correctness check or benchmark.
follow and save to keep up with the series.
AI Performance Engineering
curated by Wafer
github.com/wafer-ai/gpu-perf…
Understanding PTX, the Assembly Language of CUDA GPU Computing
by Tony Scudiero and Jonathan Bentz, NVIDIA
developer.nvidia.com/blog/un…
steve retweeted
join the @wafer_ai team for poker, food, and drinks at the University of Chicago!
we’d love to meet people doing exceptional things and hear what you’re working on.
October 28 · 5–9pm CDT · UChicago
request to join below
we’re partnering with @youdotcom to bring Wafer’s inference to developers through its new Inference Router.
access Wafer through one OpenAI-compatible API, with multi-provider failover and built-in web search from You.com.
link in thread 🧵
Introducing Inference Router: enterprise-grade inference on demand.
Kimi K3, GLM 5.3, DeepSeek V4.1 Flash, Qwen3.8 Flash, Muse Spark 1.3, Mercury 3 and more, available with an uptime SLA of at least 99.9%. US hosting and web search built in.
you.com/router
steve retweeted
we’re partnering with @youdotcom to bring Wafer’s inference to developers through its new Inference Router!
access @wafer_ai through one OpenAI-compatible API, with multi-provider failover and built-in web search from You.com.
link in thread 🧵
Introducing Inference Router: enterprise-grade inference on demand.
Kimi K3, GLM 5.3, DeepSeek V4.1 Flash, Qwen3.8 Flash, Muse Spark 1.3, Mercury 3 and more, available with an uptime SLA of at least 99.9%. US hosting and web search built in.
you.com/router
steve retweeted
we’re partnering with @youdotcom to bring Wafer’s inference to developers through its new Inference Router.
access Wafer through one OpenAI-compatible API, with multi-provider failover and built-in web search from You.com.
link in thread 🧵
Introducing Inference Router: enterprise-grade inference on demand.
Kimi K3, GLM 5.3, DeepSeek V4.1 Flash, Qwen3.8 Flash, Muse Spark 1.3, Mercury 3 and more, available with an uptime SLA of at least 99.9%. US hosting and web search built in.
you.com/router