Reports
AI-generated structured vendor updates
AMD's Experimental Topological Ghost Protocol Boosts MI300X Inference 10x
AMD introduces experimental Topological Ghost Protocol (TGP) on MI300X GPUs, achieving 431 tokens/sec with 100% success in high-concurrency inference, 10x improvement over standard vLLM. TGP uses KV-cache recycling and segmented state management, still experimental but potentially redefining AI inference benchmarks.
NVIDIA Rigel Core: Single-Threaded CPU as the New Control Plane for Agentic AI
NVIDIA unveils Rosa CPU architecture with custom Rigel core (Arm v9.2), targeting single-threaded performance for Agentic AI workloads, paired with Feynman GPU (1.6nm, 50 PFLOPS) in 2028. This shifts CPU design from core-count scaling to serial-latency optimization, directly challenging AMD EPYC and Intel Xeon dominance.
NVIDIA Vera CPU: Max Single-Threaded Performance at Scale for Agentic AI
NVIDIA launches Vera CPU, a max single-threaded CPU at scale for agentic AI. With Olympus cores delivering 1.8x sustained per-core performance over x86, 1.2TB/s LPDDR5X bandwidth, and 3.4TB/s core-to-core bandwidth, Vera integrates into NVIDIA's unified AI factory architecture, aiming to lock users into its ecosystem.
AI Innovators Adopt NVIDIA Vera — Why Max Single-Threaded CPU at Scale Matters
...
OpenAI Slashes Inference Costs 50%, Runs ChatGPT on Hundreds of GPUs via System-Level Optimization
OpenAI reduces AI inference costs by over 50% through system-level optimizations: model quantization (FP16 to INT4/INT8), KV-Cache optimization, dynamic batching, and speculative decoding. Using only hundreds of NVIDIA GPUs to serve ChatGPT's unlogged-in traffic, inference gross margin jumps from 38% to 65%, nearing breakeven.
NVIDIA Shifts AI Infrastructure Metric from FLOPS to Cost Per Token
NVIDIA advocates for "cost per token" as the primary economic metric for AI infrastructure, replacing "FLOPS per dollar." This shift moves the focus from computational inputs to business outputs, requiring full-stack optimization across hardware, software, and networking to lower enterprise AI inference TCO.