NVIDIA GB300 NVL72 Dominates AgentPerf: 20x More Agents per Megawatt Reshapes AI Inference
Summary
Key Takeaways
Artificial Analysis launched AgentPerf, the first benchmark dedicated to agentic-AI infrastructure. It runs the frontier MoE model DeepSeek V4 Pro on real coding-agent trajectories, stressing systems with dozens to hundreds of chained LLM and tool calls. NVIDIA's GB300 NVL72, a rack-scale system with 72 GPUs, topped the benchmark, delivering up to 20x more agents per megawatt than an H200, tested at 20 and 60 tokens per second per agent. Providers like Baseten, DeepInfra, and Together AI already serve agentic workloads on Blackwell. Results published June 12, 2026.
Why It Matters
NVIDIA's AgentPerf benchmark is a defensive move against AMD, Intel, and Cerebras, redefining inference value around agents-per-megawatt to lock customers into Blackwell's power efficiency. It hides real-world costs: GB300 NVL72 requires liquid cooling and >100kW per rack, forcing expensive data center retrofits. Agentic workloads expose PFC/ECN congestion bottlenecks and tail latency under multi-agent concurrency despite high HBM3e bandwidth. The CUDA ecosystem and NVLink create deep lock-in, making cross-platform migration costly and risky.
PRO Decision
【Vendors (AMD, Intel, Cerebras)】Create your own agentic benchmark emphasizing agents-per-dollar or agents-per-watt in air-cooled racks. Attack GB300's liquid cooling dependency and high TCO. Showcase open software stacks (ROCm, OneAPI) to reduce lock-in. 【Enterprises (CIOs, Architects)】Audit AgentPerf with zero trust: demand real-world power/cooling data and independent tests. Compare TCO for mixed workloads. Avoid NVLink/CUDA lock-in by evaluating cross-platform portability. 【Investors】AgentPerf strengthens NVIDIA's moat, but watch competitors' catch-up and data center retrofit costs limiting GB300 adoption. Diversify into white-box inference and cloud custom chips.
Get 3-5 key AI infrastructure signals weekly →
💬 Comments (0)