Mark Hogan

🚨 BIG NEWS: $NBIS NEBIUS IMPROVES AI TRAINING THROUGHPUT ON 192 NVIDIA B300 GPUs WITHOUT ADDING A SINGLE GPU. This is a pretty bug technical update from Nebius. Nebius tested NUMA-aware process affinity on a DeepSeek V3 pretraining workload running across 192 NVIDIA B300 GPUs using Megatron Bridge on its managed Kubernetes platform. The results? Mean training step time decreased by 1.11%, while model throughput per GPU increased by 1.13%. Now, 1.13% might not sound impressive initially. But when you're operating thousands of expensive GPUs, even a small improvement in utilization can have meaningful economic value. Modern GPU servers have multiple CPU sockets and NUMA domains. Each CPU has faster access to its local memory and PCIe connected devices. When training processes get scheduled across the wrong NUMA domains, they can introduce unnecessary memory traffic, latency and communication overhead. Nebius tested exclusive NUMA binding, which keeps processes closer to their local CPU, memory and GPU resources. The important part is that they're improving performance through infrastructure level optimization rather than buying additional compute. Imagine a hypothetical 100,000 GPU deployment achieving the same 1.13% throughput improvement. That's roughly the training throughput equivalent of adding 1,130 GPUs, assuming the improvement scales consistently. Of course, this benchmark doesn't prove the same gains will hold across every model, cluster or workload. Nebius is operating in a market where GPUs are extremely expensive, power availability is constrained and customers care about how quickly they can train models. If Nebius can consistently extract more useful compute from the same hardware, customers can finish training workloads faster while Nebius potentially improves the competitiveness and economics of its cloud platform. This wasn't a synthetic GPU benchmark. Nebius used a DeepSeek V3 pretraining workload with Megatron Bridge, which makes the result more relevant to actual large model training. This is what separates simply renting NVIDIA GPUs from building a serious AI cloud platform. Anybody with enough capital can purchase GPUs. Getting those GPUs to operate efficiently across CPU topology, memory, networking, containers and distributed training software is an entirely different engineering problem. Nebius is showing that it's paying attention to that layer and as its GPU infrastructure scales, these seemingly small optimizations could become increasingly important to its customers and its own infrastructure economics.