Nvidia GH200: Grade Hopper gets an HBM3e upgrade

The H100 GPU gets another speed boost: Nvidia launches it with HBM3e instead of HBM3 memory components. This increases both the RAM capacity and the transfer rate and is intended to accelerate the training of large AI models in particular.

Advertisement

In its announcement for the Siggraph trade fair, Nvidia speaks of a boost by a factor of 3.5 – of course in selected “best case” scenarios. For the time being, however, the improved H100 GPU is only available as a GH200 “super chip” in combination with Nvidia’s own ARM processor Grace on a common circuit board.

141 GB HBM3e per GPU

In this variant, the accelerator uses 141 GB of HBM3e memory with a transfer rate of around 5 TB/s. Nvidia does not provide any technical background to this crooked capacity – some memory locations are probably disabled. Samsung, SK Hynix and Micron are initially producing components with a capacity of up to 24 GB – six of which would actually make 144 GB.

In the original H100 version, five of the six attached HBM3 building blocks are active. 80 GB of RAM with a transfer rate of 3.35 TB/s can therefore be used. With the H100 NVL, the sixth HBM3 module is active. This increases the capacity to 94 GB – individual storage layers are also switched off here – and the transfer rate to 3.9 TB/s.

GPU H100 H100 NVL H100 (GH200 board) Shader cores 14,592 14,592 14,592 Memory 80 GB HBM3 94 GB HBM3 141 HBM3e Transmission rate 3.35 TB/s 3.9 TB/s approx. 5 TB/s

The rest of the equipment does not change between the three versions. They all use 14,592 shader cores, which achieve a computing power of almost 4 petaflops during AI training. This value applies to FP8 calculation with low precision, but sufficient for training.

Advertisement

With 72 Neoverse V2 ARM cores, the Grace CPU of the GH200 boards is now part of the upper middle class. The main computing load is by far the H100 GPU.

Not available yet

The first systems with the new GH200 boards should be available in the second quarter of 2024. By then, the memory manufacturers should be able to deliver HBM3e components. In addition, Nvidia still has time to ease the delivery bottlenecks.

Systems with the previous Grace Hopper combination are still rare. Nvidia announced it in early summer 2022 for the first half of 2023; the first models were on display at the Computex trade fair in May. However, they have not yet appeared commercially.

(mma)

Home
#Nvidia #GH200 #Grade #Hopper #HBM3e #upgrade