The Nvidia Quadro M5000 is a workstation grade GPU. It uses the same chip as the Titan X (a consumer grade card that costs around $1000) so it will perform about the same as the Titan X. The Quadro line is fully designed, manufactured and tested by Nvidia so it has very good compatibility with professional software. It costs around $2500 which is slightly over your budget but is a card you should look at.
The Nvidia Quadro M4000 is a tier below the M5000. It uses the same chip as the GTX 970 (which is around $350). It costs round $1000. If you got this I would recommend you to get 2 of them so you can run them independently and split the VMs between these two cards. This can be used as long as there is not single VM that require a insane amount of processing power.
You can also choose to use the consumer line of GPUs, I recommend getting 2 x Titan X if you need the extra VRAM but I would recommend 2 x GTX 1080 otherwise. You can run these in SLI if you want them to act as a single faster card or run them independently (I recommend this way).
If you want more performance, then go with the consumer GPUs. If you care more about stability and comparability, then go with the Quadro.