CDNA3 GPU with massive 192GB HBM. 1.3K TFLOPS FP16 for production-grade compute. 169K tok/s MLPerf.
Metrics
Memory GB192
High-bandwidth memory capacity
TFLOPS FP161,307
FP16 compute performance
ArchitectureCDNA3
Neural network architecture type
TDP750W
Thermal design power consumption
Score Breakdown
training
15%51
LLM training time-to-convergence
ecosystem
15%87
Software stack maturity and framework support
inference
25%95
LLM inference throughput per GPU
performance
30%88
Raw compute throughput and memory bandwidth
availability
15%80
Market supply and cloud instance availability
Compatibility(3)
Sources(1)
Developed by
AMDRelated Signals(2)
MI300X Leads MLPerf Inference v5.1 for LLM Throughput
AcceleratorsJan 10
94AMD MI300X achieves highest per-GPU LLM inference throughput in MLPerf Inference v5.1, delivering 21,150 tokens/s per GPU on llama2-70b, outperforming NVIDIA H100 (15,610 tok/s), B200 (13,015 tok/s), and H200 (10,917 tok/s). Industry-standard benchmark validates AMD's competitiveness in AI inference.
Related:
AMD MI300X Gains Enterprise Traction
AcceleratorsDec 10
82AMD's MI300X is seeing increased adoption as enterprises seek alternatives to NVIDIA's supply-constrained GPUs, with 192GB memory enabling larger model deployments.
Related: