I am using Keras with Theano backend. Until now I have used AWS EC2 GPU instances for train my keras network. As my experiments become larger, my network trains became longer and my bills became bigger. I calculated that good workstation would be a better investment than renting AWS EC2 GPU instances on the cloud. I am copying components of NVIDIA DIGITS DevBox, except there are new GPUs on the market, so I have couple of question about them.
Is it better, for deep learning, to buy four GTX 1080 or dual Titan X Pascal (specs)? Four GTX 1080s give me 10240 CUDA cores with boost speed @1733 Mhz, 32GB of GDDR5X VRAM and 36 TFLOPs of FP32 Compute. For the same price dual TITAN X Pascal will give me 7168 CUDA cores boost clock @ 1530 Mhz, 24 VRAM GDDR5X, and 22 TFLOPs of FP32 Compute.
My plan of using multiple GPUs is train same model with different parameters on different GPUs, to see which combination of parameters is most effective. I need the GPU solution that is most efficient for this type of usage.