eight or even 16 A100 GPUs that can be interconnected with Nvidia’s NVLink and NVSwitch interconnect technology, which provides nearly 10 times the bandwidth of PCIe 4.0, according to Kharya.

The Ampere server could either be eight GPUs working together for training, or it could be 56 GPUs made for inference,' Nvidia CEO Jensen Huang says of the chipmaker's game changing A100 GPU.