Wenhao Ding

What if AI could improve not only its own policies, but also the judges that evaluate them? As AI policies improve, they encounter new challenges and develop new failure modes. The judges evaluating them need to evolve as well. Otherwise, they can become a bottleneck to further progress. This is particularly relevant for #Physical #AI, where good decisions require spatial grounding, causal reasoning, and safety awareness. In our new paper, we introduce VeriFine, an #agentic #harness for continuous self-improvement in Physical AI, where policies, training curricula, and judges evolve together. The idea is to build agentic systems that can identify policy failures, turn them into useful training signals, and improve the policy. When the judge itself becomes a bottleneck, the harness brings in targeted human feedback to refine its evaluations and guide the next round of improvement. We evaluate VeriFine on autonomous driving and robot navigation, demonstrating continued improvements in both policy and judge capabilities through reinforcement learning and supervised fine-tuning. Key takeaway: We're developing agentic harnesses that go beyond optimizing a policy in a fixed loop. They adapt the improvement process itself—including how failures are identified, how training data is selected, and how progress is evaluated. 📄 Paper: arxiv.org/pdf/2610.08761 🌐 Website: veri-fine.github.io/ Work led by Zewei Zhou, in collaboration with Rachel Luo Yulong Cao @ChaoweiX Chensheng Peng @Boyiliee @tianran_ Zheng Lian @yan_wang_9 Jiaqi Ma @iamborisi @wenhaoding95 #PhysicalAI #Robotics #AgenticAI #Alpamayo @NVIDIADRIVE @NVIDIAAI

How do we make robot policies robust to rare but high-impact failures? Video #World #Models (WMs) are rapidly becoming a powerful tool for robotics, enabling policy evaluation and improvement by "imagining" future outcomes. But there's a catch: these imagined futures are typically nominal samples, making it easy to overlook the rare yet safety-critical events that matter most. In our new paper, StressDream: Steering Video World Models for Robust Policy Evaluation and Improvement, we explore a simple but powerful idea: 💡 Instead of passively sampling futures, actively steer world model imaginations toward high-impact yet still plausible scenarios. StressDream optimizes the initial diffusion noise at inference time, allowing us to generate targeted stress-test scenarios without retraining the world model. This enables: - More robust policy evaluation by exposing failure modes that random sampling often misses. - Improved policy optimization by training against challenging but realistic imagined futures. As generative world models become a foundation for #Physical #AI, the ability to systematically probe their "long tail" of plausible futures will be increasingly important for building reliable and trustworthy autonomous systems. 📌 𝖯𝗋𝗈𝗃𝖾𝖼𝗍 𝖯𝖺𝗀𝖾: junwon.me/StressDream/ 📄 𝖯𝖺𝗉𝖾𝗋: arxiv.org/abs/2606.00267 Work led by Junwon Seo, with a great set of collaborators: Sushant Veer, Thomas Ran Tian, Wenhao Ding, Apoorva Sharma, Karen Leung, Edward Schmerling, Andrea Bajcsy. @NVIDIADRIVE @NVIDIAAI #Robotics #WorldModels #PhysicalAISafety #AISafety #AutonomousSystems #RobotLearnin

Today we're releasing @nvidia AlpaGym, our new open-source reinforcement learning (RL) framework for end-to-end autonomous driving. A key challenge for #Physical #AI is enabling policies to learn from the consequences of their actions. While supervised learning can teach a model to imitate behavior, robust autonomy ultimately requires learning through interaction with the environment. AlpaGym enables exactly that. Built on top of: - AlpaSim: our high-fidelity closed-loop autonomous driving simulator - Cosmos-RL: NVIDIA's distributed RL training and rollout infrastructure AlpaGym provides the glue that connects simulation, training, and driving policies into a scalable framework for post-training autonomous vehicle models in closed loop. With AlpaGym, researchers and developers can: ✅ Train end-to-end driving policies using reinforcement learning ✅ Run large-scale closed-loop simulations ✅ Experiment with new reward functions, policy architectures, and training strategies ✅ Benchmark models on public leaderboards 📖 Learn how it works: developer.nvidia.com/blog/ho… 💻 GitHub: github.com/NVlabs/alpagym 🏆 Open Challenges: - AlpaSim Closed-Loop E2E Driving Challenge: huggingface.co/spaces/nvidia… - Physical AI AV Reasoning Challenge: huggingface.co/spaces/nvidia… Learn more about the #Alpamayo open platform: huggingface.co/blog/drmapavo… #PhysicalAI #AutonomousDriving #ReinforcementLearning #Robotics #OpenSource #NVIDIA #MachineLearning @NVIDIADRIVE @NVIDIAAI

What does it take to build autonomous vehicles that can reason about the world they drive in? Tomorrow at #NVIDIAGTC, Patrick Liu and I will take a deep dive into the #Alpamayo #reasoning model family—a family of reasoning-based vision–language–action (#VLA) models that form a core component of the Alpamayo open platform (huggingface.co/blog/drmapavo…). We’ll cover three main topics: - How reasoning-based VLA models like Alpamayo 1 are designed and built - What it takes to bring Alpamayo 1 to production, including some of our latest results - Several exciting announcements about the expansion of the Alpamayo open platform If you're working on autonomous driving, robotics, or foundation models for physical AI, this session will offer a look at where the field is heading. Session details: 📅 Monday, Mar 16 | 3:00 PM PDT 📍 #NVIDIAGTC 2026 🔗 nvda.ws/4rze5oj Looking forward to seeing many of you there. @NVIDIADRIVE @NVIDIAAI

Join me and my collaborators for a *live* discussion on @nvidia Alpamayo 1 (huggingface.co/nvidia/Alpama…), a reasoning-based vision–language–action (VLA) model for autonomous driving. 🎥 Livestream: Inside NVIDIA Alpamayo 1: Making Autonomous Vehicles Reason 🗓 February 11 ⏰ 9:00am PST 📍 Watch here: youtube.com/watch?v=V9E4GX5v… As NVIDIA CEO Jensen Huang put it: “The ChatGPT moment for physical AI is here — when machines begin to understand, reason, and act in the real world. Robotaxis are among the first to benefit. Alpamayo brings reasoning to autonomous vehicles, allowing them to think through rare scenarios, drive safely in complex environments, and explain their driving decisions — it’s the foundation for safe, scalable autonomy.” During the livestream, we’ll cover: - How #reasoning-based #VLA models like #Alpamayo 1 are designed and built - Applications ranging from end-to-end #autonomy to reasoning-driven auto-labeling - Key opportunities and challenges in developing reasoning models for #Physical #AI I’ll be joined by core Alpamayo 1 developers @yan_wang_9 @YurongYou @wenhaoding95, and we’ll take questions live from the community. 📖 Ahead of time, you might enjoy this overview of the Alpamayo ecosystem: huggingface.co/blog/drmapavo… And if you’re attending @NVIDIAGTC (March 16–19) and would like to meet some of the Alpamayo team in person, you can use my employee code for 25% off your conference pass: nvidia.com/gtc/?ncid=GTC-NV1… Hope to see you at the livestream! @NVIDIAAI @NVIDIADRIVE

It’s incredibly exciting to see how quickly the community is engaging with the @nvidia Alpamayo ecosystem for developing reasoning-based autonomous vehicles (huggingface.co/blog/drmapavo…)! In this instance, TIER IV is showcasing Alpamayo 1’s reasoning capabilities in Tokyo, integrated with Autoware and ROS. Fantastic work, @ShinpeiKato and the @tier_iv_global team! 👏 Quick highlights about Alpamayo: Alpamayo 1: - Among HuggingFace’s top 10 overall trending models - Among the top 3 most downloaded models on HuggingFace when filtered by 'robotics' Alpamayo PhysicalAI–Autonomous-Vehicles dataset: - Trending in HuggingFace’s top 10 overall datasets Happy developing! 🚀 #AutonomousVehicles #Robotics #AI #Reasoning #HuggingFace #Autoware #ROS #AutonomousDriving #PhysicalAI #Alpamayo #RobotLearning @NVIDIAAI @NVIDIADRIVE

More on #reasoning in Vision-Language-Action (#VLA) models --- Traditional VLA models decide what action to take by decomposing complex situations into their most salient factors. But reasoning models can do much more. When viewed as implicit world models operating in a semantic space, they can be used counterfactually—exploring multiple “what if” scenarios before acting. In our recent paper, Counterfactual VLA (CF-VLA, arxiv.org/pdf/2512.24426), we show that counterfactual reasoning consistently improves trajectory accuracy, safety, and reasoning quality. Key contributions: - Self-reflective counterfactual reasoning: CF-VLA reflects on predicted meta-actions, anticipates consequences, and revises plans before execution—enabling causal self-correction. - Automated data pipeline: A novel data pipeline generates counterfactual data, forming a self-improving loop for reasoning and action. - Adaptive thinking in autonomous driving: CF-VLA focuses reasoning on the most challenging scenarios, improving performance while keeping test-time computation efficient. Paper: arxiv.org/pdf/2512.24426 #AI #Robotics #VisionLanguageAction #AutonomousSystems #MachineLearning #CounterfactualReasoning @NVIDIAAI @NVIDIADRIVE

🚗 Imitation learning is everywhere—but is it enough? So far, imitation learning—most commonly via behavior cloning (BC)—remains the go-to approach for training real-world autonomous vehicle (AV) driving policies. Yet BC operates in an open-loop (OL) fashion, overlooking the critical interdependence among inputs, outputs, and future states that comes with closed-loop (CL) operation. The result? The notorious—but often overlooked—OL–CL gap ⚠️ To address this challenge and encourage broader adoption of CL techniques, we’ve just published a survey (research.nvidia.com/publicat…) presenting a comprehensive taxonomy of closed-loop training methods for end-to-end driving. Our framework organizes approaches along three key axes: - Action generation - Environment response generation - Training objectives 💡 Bottom line: enabling technologies—like neural rendering, generative world models, and scalable RL—have now matured, making closed-loop AV training ready for wide-scale adoption. We’d love to hear your thoughts—drop a comment and join the discussion! 💬 And as a reminder, we are hiring for full-time research scientist and research engineer positions: 🔹 [Sr.] Research Scientist: nvidia.wd5.myworkdayjobs.com… 🔹 [Sr.] Research Engineer: nvidia.wd5.myworkdayjobs.com… @NVIDIADRIVE @NVIDIAAI @nvidia

Excited to unveil @nvidia's latest work on #Reasoning Vision–Language–Action (#VLA) models — Alpamayo-R1! Alpamayo-R1 is a new #reasoning VLA architecture featuring a diffusion-based action expert built on top of the #Cosmos-#Reason backbone. It represents one of the core technologies driving NVIDIA’s push toward Level 4 autonomy and robotaxis (nvidianews.nvidia.com/news/n…), as announced by Jensen Huang at #gtc DC last week. 📄 Paper: Alpamayo-R1 research.nvidia.com/publicat… We present: - Architecture & Design: How to transform a VLM into a driving-ready Reasoning VLA - Chain of Causation Labeling: A new framework enabling reasoning-based learning - Training Strategy: From internet-scale pre-training → AV-specific SFT → RL-based post-training - Extensive Evaluation: From closed-loop simulation to real-world, on-vehicle testing 📈 Results: Alpamayo-R1 delivers significant performance gains over end-to-end baselines — especially in rare, safety-critical scenarios — all while maintaining real-time inference (99 ms end-to-end latency). Coming soon: releases of model variants and reasoning metadata built on top of the Physical AI Dataset (huggingface.co/datasets/nvid…)—with more updates on the way. Stay tuned! 🙌 Huge thanks to Wenjie Luo and @yan_wang_9 (project co-leads); the @nvidia AV Research team (@iamborisi, @YurongYou, @xinshuoweng, @tianran_, @wenhaoding95, and many others); collaborators across @nvidia Research (@liu_mingyu, @visualyang, @PavloMolchanov, and many others); and the @nvidia AV Product team (Sarah Tariq, Patrick Liu, Jack Huang, and many more). Full contributor list in the Appendix. @NVIDIADRIVE @NVIDIAAI

🚗🤖 Interested in reasoning models for embodied AI? I’m excited to share that at #NVIDIAGTC in DC I’ll unveil our latest work at #NVIDIA on reasoning Vision-Language-Action (VLA) models for vehicle autonomy: I’ll cover how we’re: • Advancing reasoning in VLA models • Powering a data flywheel for AV foundation models • Making autonomous driving more human-like and safer — with real-world driving videos! 🔗 Session info: nvda.ws/3IGfvwq 📍 Tuesday, Oct 28 • 3 PM Walter E. Washington Convention Center

The Autonomous Vehicle (AV) Research Group @NVIDIAAI is looking for talented interns! Dive into cutting-edge work—from reasoning models and generative simulation to AI safety—and help shape the future of AV and embodied AI. Ready to push the limits? Apply now: nvidia.wd5.myworkdayjobs.com…

We’re now accepting applications for the 2026–2027 NVIDIA Graduate Fellowships! If you’re passionate about advancing cutting-edge reasoning models for Physical AI applications 🚗🤖, apply here: research.nvidia.com/graduate… — and be sure to select “Autonomous Vehicles.” @NVIDIAAI

Can we use simulation to validate Physical AI? Yes—with far fewer real-world tests. We propose a control variates–based estimation framework that pairs sim & real data to dramatically cut validation costs. #AI #Robotics #Sim2Real" Paper: arxiv.org/pdf/2506.20553 @NVIDIADRIVE