How to build two GPU servers in one?

Question

Is the NVIDIA catalog incorrect?: NVLink is a connector that is more widely available than the catalog suggests. Either way, NVlink only connects 2 PCIe4 cards and I am looking to build a server with this.

The server becomes a little schizophrenic from here on.

On the one hand I want it to host two (A100) GPU's, connected with NVLink. However, these GPU's will only be on a VPN, not the internet. So these GPU's should have their own CPU, RAM, storage, VPN network hardware. If you have in depth knowledge, I would like to be able to monitor what goes in and out of this subsystem. Is this possible?

On the other hand, I do want to also host one or two A2 (or T4) GPU's. These GPU's are part of a web server, with the required network and storage hardware.

Is this configuration possible with a single ASUS ESC4000A-E10? Or with any server you know of? A blade architecture is not possible.

Answer 1

I recently got a server with 2 A100s, but they take up so much power, that you can't reasonably put any more GPUs in the same system. I already had to run a 208V line just for this server. Even if you have the power supply for it, you probably can't fit the cooling. You can split them up into 8 parts each, if you pay Nvidia a licensing fee for the privilege: Nvidia Grid.

If you have to split up the GPUs you can pass them through to Virtual Machines. If your doing this anyway, I would strongly recommend getting two identical servers and setting them up to migrate VMs between themselves as needed. You don't have to buy twice what you need (fully redundant) you can just split the number of cards you need between the two machines.

NVlink is for data centers and their crazy configurations. If you only need 2 A100, you can just use the PCIe version of the cards.