Expert Takeaways: Inference > GPU Rental Rates Up ~20%: Driven by HBM supply constraints. Neo-cloud providers are passing hardware cost increases straight to tenants via shorter 1–3 year contracts rather than taking margin cuts. > HBM4 Bottlenecks Capping NVIDIA Vera: NVIDIA’s Vera GPU faces manufacturing constraints—HBM4 issues restrict output to 100k–200k units vs. its 400k–600k theoretical capacity. > Low-Latency ASICs Gaining Share: Agentic AI is splitting workloads into general vs. ultra-fast inference. Low-latency hardware currently sits at 5–10% market share and is projected to reach ~40% long-term. > Etched Hits ~60k Tokens/sec: By stripping general-compute circuitry, Etched boosts transformer utilization to ~90%. It has raised $10B in fresh capital with deployments at Jane Street & JPMorgan. > Groq Commercial Strategy: Offering 50% discounts on cached prompts and batch APIs. LPX service baseline priced at $0.05–$0.09 per million input tokens, with native integration for Rubin. > Groq LPU Benchmarks across Kimi Models: Kimi 1.5 reaches ~1,000 tokens/s, Kimi K2 delivers 400–450 tokens/s, and multi-LPU Kimi 3 hits ~250 tokens/s. > Cloud Benchmark Breakdown (Kimi K2.5): Together AI hits 450 tokens/s (1.37s latency), Fireworks AI leads on speed with sub-second latency (354 tokens/s), and standard neo-clouds trail at 338 tokens/s (1.86s latency). $NBIS $NVDA $MU $CBRS