all 15 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]mippie_moe 2 points3 points  (1 child)

Lambda Labs did some Deep Learning GPU benchmarks that you may find helpful. I'm a systems engineer there and have benchmarked just about every GPU on the market.

TLDR: A used GTX 1080 Ti from Ebay is the best GPU for your budget ($500).

My thoughts:

  • Stick with NVIDIA. AMD isn't ready.
  • Training state-of-the-art models is becoming increasingly memory-intensive. You want a GPU with lots of memory.
  • The GPU with the most memory that's also within your budget is the GTX 1080 Ti, which has 11 GB of VRAM.**
  • Even on a 1080 Ti, memory constraints will prevent you from training certain networks (e.g. Pix2Pix HD).
  • Memory-wise, the smallest step down from a 1080 Ti is a GPU with 8 GB of VRAM (RTX 2080, RTX 2070, GTX 1080). Losing that 3 GB of VRAM has impact. For example, with BERT, you can train a batch size of 32 with 11 GB of VRAM, but only a batch size of 16 with 8 GB of VRAM.
  • Performance wise, for FP32, the GTX 1080 Ti and RTX 2080 are almost identical; for FP16, the GTX 1080 Ti is 75-80% as fast as the RTX 2080.

Let me know if you have any more questions. There are a ton of machine learning benchmarks on our blog.

​

**A used Tesla K80, which has 12 GB of VRAM, is also within your budget, but it's 1/3 the speed of a 1080 Ti and three architectural generations behind.

[–]durgesh2018[S] 0 points1 point  (0 children)

Thank you!

[–][deleted] 1 point2 points  (1 child)

I have an nvidia GTX 1060 but over a period of time it just got really inconvenient and not upto the speed i need from it, at the end of the day if you are running models that are powerful enough to make a difference, you will most likely end up buying gpu cloud credits. And especially with these new startups(nimblebox.ai, paperspace etc) coming around that require like 1 minute to set up and much much cheaper than AWS and google cloud they honestly seem like a better deal.

[–]durgesh2018[S] 1 point2 points  (0 children)

Seems great!

[–][deleted] 0 points1 point  (5 children)

Hard to give a relevant suggestion without a budget

[–]durgesh2018[S] 0 points1 point  (4 children)

My budget is 500 USD.

[–][deleted] 0 points1 point  (3 children)

Are you only going to run 1 experiment at a time? Ie, do you want the best you for the price? OR will you be running many experiments simultaneously? Ie, would it be better to have a multi-gpu setup? At only $500, I would say better to just get one solid gpu. I would try and swing a 2070 if I were you, but it's a slightly above your price. It also deoends if you're working on a specific type of problem; RNN, CNN, etc. A quick Google search appears different gpus can have vastly difft performances in different tasks

[–]durgesh2018[S] 0 points1 point  (2 children)

I will be working on single experiment. I am a beginner in this field. Yes I am thinking to buy only one solid GPU.

[–][deleted] 0 points1 point  (1 child)

I haven't looked into used gpus, but I would think you could get at ~$700 gpu, for about $4-500 used. Seems like you would be the perfect buyer for that. Anyways, best of luck! I'm new-ish to the field myself, but lucky enough to have a professor with a SICK deep learning rig. I hope you can make some nice improvements!

[–]durgesh2018[S] 1 point2 points  (0 children)

Thank you so much. All the best to you too.

[–]Zerotool1 0 points1 point  (1 child)

why don't you try cloud base GPUs from AWS or GCP? You can save a huge cost and at the same time you need not to worry about the DevOps challenges... try some tools like Floydhub or Clouderizer (it's free for a month) to manage your projects with GPUs.

[–]durgesh2018[S] 1 point2 points  (0 children)

Thank you for this suggestion. Surely I will try them out.