I am looking forward to invest in a good GPU/ASIC for Machine-Learning research. I do not want to choose any of NVIDIA's consumer-grade GPUs as they are mainly gaming oriented and the price I'd be paying wouldn't be justified as I won't be using any game-play oriented features like Ray Tracing. And also, the previous generation GPUs' prices have gone up despite the launch of new 20XX series making the 10XX not a good buy.

AMD's consumer-grade GPUs aren't still powerful enough as their counterparts and their enterprise-grade GPUs are too costly and don't justify the price to performance ratio.

Recently, Habana Labs announced their ML oriented chipset but its samples ship Q2 of 2019 and I might consider waiting if it is justified.

So, are there any Good consumer grade ASIC/GPUs for ML?

P.S: I don't mind library support

  • The Habana chip is a lot faster than Nvidia's V100, what is your budget?
    – Rob
    Jun 17 '19 at 9:41

The quick answer:

No, there are no consumer grade ML ASICs you can purchase, and there won't be until the one mentioned in your post launches.

The rabbit hole:

If you absolutely need local hardware:

Most GPUs still have an amazing value proposition for ML, even if they don't have tensor cores. Generally speaking, Nvidia GPUs are more supported in the ML world, so unless you physically can't, go with Nvidia for this task. Right now, the RTX 2080/2080ti, even with ray-tracing not utilized, is the best value proposition for money down, in your machine hardware, thanks to their tensor cores. The next step up would be the 5000$ GV Titan card, which is in a price category you want to avoid. If you really don't want to waste any potential, grab a high-end 10 series card, but be aware that these do not have any ML hardware optimizations.

The ASIC-like option:

Your question made me remember that Google mentioned some time ago that they too were developing ML orientated ASICs called TPUs short for "tensor processing units". I did some digging and it seems that this project has morphed from commercially available ASICs into a ML research driven cloud service. I will link it here: https://cloud.google.com/tpu/docs/pricing

If you are willing to use a cloud service, your value proposition changes depending on how often you have to train your ML algorithms. If you are going to train your algo 24/7 with minor tweaks, go with a local GPU. Otherwise the Google service seems like a reasonable alternative to a hardware investment.

  • What do you mean when you say 10XX cards are not optimized for ML? Can you add some stats to your answer? AFAIK RTX2080Ti is 1.5x powerful in general gaming benchmarks, but can't say anything about ML benchmarks
    – Dhruva
    Sep 27 '18 at 19:36
  • Note I said hardware optimizations, as far as I am aware, no 10 series card ship with tensor cores. I will link a Tom's Hardware article that discusses Nvidia's findings about performance on the V 100 die, which has tensor cores, vs the p100 die, which does not. The 2080ti's TU102 die has slightly less tensor cores than the v100(576 vs 640) so the results from a 2080ti should be comparable , though slightly less than, a gv100 based Titan card. Link: tomshardware.com/news/nvidia-tensor-core-tesla-v100,34384.html
    – user9313
    Sep 27 '18 at 21:11

Consider Intel's (former Altera) FPGAs - they have an OpenCL compiler. They even have PCIe development kits. Or the Xeon Phi series which might be better suited for this.

  • Xeon Phi has relatively low performance. Xeon Phi or any other Intel's FPGAs wouldn't suit my rather heavy workload
    – Dhruva
    Sep 25 '18 at 13:24

The closest to what you are asking is NVIDIA Tesla - it's dedicated, does not have even single DP / HDMI output, it's power efficient and is optimized for 'AI'. See NVIDIA Tesla T4, NVIDIA Tesla V100, NVIDIA Tesla P100, NVIDIA Tesla P4/P40. Whether it's most cost efficient depends on many factors. If you're buying for a larger institution, take a look at data center solution based on Tesla V100.

The second candidate would be NVIDIA Jetson. It looks to be designed for ML / AI in robotics, as an end-to-end solution (as opposed to agent communicating with server). It's a dev kit, including ARM CPU, GPU, RAM, eMMC storage and 2 DeepLearning accelerators, at a price tag of 2.5k $ (but it's small!). I could not find any benchmarks, but they state, that it's 20 times faster and draws 1/10 of electricity than it's predecessor, NVIDIA JETSON TX 2, which was benchmarked ex. here.

Having said that, it really depends on your use cases - if you will profit from it's modularity, dont' have any other hardware bought for the setup and will use it as embeded (like you'd use RaspberryPi), that NVIDIA's Jetson is perfect. If you want to make a server for AI, I would consider RTX 2080/2080ti as user9313 suggests and NVIDIA Tesla line, and peak whichever is most cost efficient for you - which is impossible to say without knowing the framework / library you'll use. Also, as mentioned by user9313, the cloud is already a viable option, not only Google 1,2 but also Microsoft has good APIs and even cloud IDE.

Also, adding to the wish-list other projects to keep an eye on:

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.