ACM SIGGRAPH

🚀 @siggraph 2026 Day 1: future of Physical AI will not be won by bigger generative models. It will be won by explicit grounding, structured memory, and controllable generation. 1️⃣ DLSS 5 (@edliu1105): Generation needs structure @nvidia's key insight is that the rendered frame is the foundation. Instead of relying on prompts alone, DLSS 5 conditions on: • Character identity • Scene semantics • Lighting semantics • Camera/composition Combined with per-asset controls, one-step diffusion, 4K real-time inference, single-GPU deployment, and frame-streaming generation, DLSS 5 sits at the controllable, causal, low-latency end of the generative spectrum. 2️⃣ @NVIDIAAI Cosmos 3 (@liu_mingyu): World models need unified action spaces @NVIDIARobotics’s Cosmos vision unifies: World Understanding → Prediction → Simulation → Action Key ingredients: • Hybrid Mixture-of-Transformers architecture (causal AR branch + diffusion branch) • Shared multimodal attention across vision, text, audio, and action • Unified action representation (9D ego/effector pose + 15D/1D grasp state) across AVs, manipulators, humanoids, and egocentric agents • Forward dynamics, inverse dynamics, and video-action policies • Step distillation reducing inference from 50→4 steps (25× speedup) • Cosmos 3 family: Edge 4B, Nano 16B, Super 64B The most important slide may have been: Compute → Data → World Foundation Models 3️⃣ Pixal3D: Fidelity comes from correspondence Most image-to-3D systems suffer from an implicit 2D↔3D alignment problem. Pixal3D solves this through: • Pixel-aligned sparse SDF latent representations • Explicit image back-projection into 3D feature volumes • Direct pixel-to-3D correspondence • Two-stage generation (coarse geometry → latent refinement → PBR mesh) The lesson: replace ambiguous attention with explicit geometry. 4️⃣ Mixture of Contexts (@iclr_conf 2026): Memory is retrieval Long-video generation is fundamentally a memory problem. MoC reframes long-context modeling as retrieval: • Learnable sparse attention router • Each query selects a few relevant history chunks + mandatory anchors • Causal routing preserves temporal consistency • End-to-end differentiable training Results on minute-long videos (~180k tokens): • ~85% attention pruning • ~7× FLOP reduction • Near-linear scaling instead of quadratic attention • Maintained identity, action, and scene coherence The key insight: Long-term memory emerges from retrieval, not dense attention. Summary DLSS 5 → explicit scene grounding Pixal3D → explicit pixel-3D grounding MoC → explicit memory grounding Cosmos → explicit action grounding The industry is quietly moving away from the idea that scaling video generation alone will solve embodied intelligence. The next generation of world models will be defined by: ✅ Structured correspondences ✅ Retrieval-based memory ✅ Unified action representations ✅ Controllable generation ✅ Efficient inference #SIGGRAPH2026 #NVIDIA #PhysicalAI #Robotics #WorldModels #GenerativeAI #ComputerGraphics #EmbodiedAI #3DVision #VideoGeneration #DLSS #Cosmos #Pixal3D #ICLR2026