To that end, AMD has launched the MI325X with greater memory capacity and bandwidth than the Instinct MI300X, which launched last December. The MI325X is based on the same CDNA 3 GPU architecture, compared with 192GB of HBM3 high-bandwidth memory and 5.3 TB/s in memory bandwidth in the MI300X.
AMD said AI inference performance in the MI325X provides 40% faster throughput with an 8-group, 7-billion-parameter Mixtral model over Nvidia’s top-of-the-line Hopper H200, 30% lower latency with a 7-billion-parameter Mixtral model, and 20% lower latency with a 70-billion-parameter Llama 3.1 model.
AMD is planning an eight-node platform for next year, similar to Nvidia’s DGX Pods. With eight MI325X GPUs connected over AMD’s Infinity Fabric, the platform will offer 2TB of HBM3e memory, 48 TB/s of total memory bandwidth, 20.8 petaflops of FP8 performance, and 10.4 petaflops of FP16 performance, AMD said.
The MI325X will begin shipping in systems from Dell Technologies, Lenovo, Supermicro, Hewlett Packard Enterprise, Gigabyte, and several other server vendors starting in the first quarter of next year, the company said.
Read more processor news
- Enfabrica looks to accelerate GPU communication: Enfabrica’s Accelerated Compute Fabric SuperNIC (ACF-S) silicon is designed to deliver higher bandwidth, greater resiliency, lower latency and greater programmatic control to data center operators running data-intensive AI and HPC.
- Nvidia claims efficiency gains of up to 100,000X: However, the chipmaker’s dramatic claim for the performance gains of its GPUs is over a 10-year span, and only applies to one type of calculation.
- Intel launches Xeon 6 processors and Gaudi 3 AI accelerators: Intel has formally launched its next Xeon 6 server processors as well as the Gaudi 3 AI accelerators, making some pretty big boasts in the process.
- Inflection AI shifts to Intel Gaudi 3, challenging Nvidia’s AI chip lead: The announcement follows IBM’s recent partnership with Intel, signaling a rising interest in Intel’s AI hardware.
- Intel’s Altera spinout launches FPGA products, software: Altera CEO Sandra Rivera shares ‘big, audacious, ambitious goal’ to dominate FPGA market.
