Blackwell GPU with massive 192GB HBM. 4.5K TFLOPS FP16 for next-generation performance. 52K tok/s MLPerf.
Metrics
Memory GB192
High-bandwidth memory capacity
TFLOPS FP164,500
FP16 compute performance
ArchitectureBlackwell
Neural network architecture type
TDP1000W
Thermal design power consumption
Score Breakdown
training
15%95
LLM training time-to-convergence
ecosystem
15%99
Software stack maturity and framework support
inference
25%58
LLM inference throughput per GPU
performance
30%99
Raw compute throughput and memory bandwidth
availability
15%90
Market supply and cloud instance availability
Compatibility(3)
Sources(1)
Developed by
NVIDIARelated Signals(5)
MI300X Leads MLPerf Inference v5.1 for LLM Throughput
AcceleratorsJan 10
94AMD MI300X achieves highest per-GPU LLM inference throughput in MLPerf Inference v5.1, delivering 21,150 tokens/s per GPU on llama2-70b, outperforming NVIDIA H100 (15,610 tok/s), B200 (13,015 tok/s), and H200 (10,917 tok/s). Industry-standard benchmark validates AMD's competitiveness in AI inference.
Related:
NVIDIA AI Compute Stock Doubling Every 10 Months
AcceleratorsDec 31
91Installed AI compute from NVIDIA chips has more than doubled annually since 2020, with new flagship chips accounting for most compute within 3 years of release.
Related:
B200 Blackwell GPUs Enter Production
AcceleratorsDec 10
90NVIDIA's B200 Blackwell GPUs are shipping to hyperscalers, promising 2.5x performance gains over H100 for AI training workloads.
Related:
InferenceMAX Launches Open Benchmark for AI Accelerators
AcceleratorsOct 9
91SemiAnalysis launched InferenceMAX, an open-source nightly benchmark comparing GPU inference performance across NVIDIA (H100, H200, B200, GB200 NVL72) and AMD (MI300X, MI325X, MI355X). Endorsed by Jensen Huang, Lisa Su, OpenAI, and Microsoft. First multi-vendor benchmark with TCO and power efficiency metrics.
Related:
AMD MI355X Achieves Competitive TCO vs NVIDIA B200
AcceleratorsOct 1
89AMD MI355X delivers lower TCO per million tokens than NVIDIA B200 for GPT-OSS 120B FP4 summarization at interactivity below 225 tok/s/user. MI300X also beats H100 on GPT-OSS 120B MX4 across all interactivity levels. B200 leads on LLaMA 70B FP4 and high-interactivity workloads.
Related: