(I wasn't sure if this should apply more to serverfault, or here, so feel free to migrate)
I'm a longtime server admin for various HPC related projects, but in all cases so far, I've used a cloud provider of some sort. I'm familiar with general hardware deployments for many of my own projects, but nothing of this budget / scale so far.
I'm looking for some recommendations for machines for a small HPC for an academic lab where jobs will generally be either CPU-heavy, GPU-heavy, or I/O heavy (but usually not all together). Most likely ubuntu-server. I know that doesn't leave much room for compromise. The main bottleneck in most of my deployments so far has been in the I/O performance, so that's something I would like covered especially. (NVME or such, raid60?)
-I would like the main storage space to be shared by all machines. I've tried to use GlusterFS to mitigate split storage in prior deployments and performance was poor.
My main two points of confusion related to server grade hardware selection are: -For shared storage, I am looking to use something more performant than NFS if possible. What kind of hardware do I need to directly share out the storage from the storage server. iSCSI? -For GPU capability, I often see machines that have the appropriate PCI-express availability, but it's difficult to confirm whether they will support GPUs (either due to sizing, or because they lack the appropriate power leads). What would be a good rule of thumb to use to determine if GPU's are supported by a specific server hardware?
My Budget is somewhere in the range of 70K USD. Just looking to get a few ideas.
Thanks for reading.