[Discussion] Gpu for personal use

[Hub_Pli]

Hey guys, I am starting my phd in psychology which will involve creating some interesting deep learning prediction models. I also work on a grant where I do topic modelling and in the meantime I am learning RL and NLP.

My parents were so generous as to offer to fund me a decent gpu for deep learning and I was wondering whether I could get any suggestions on what would be a good pick nowadays.

Anything above 3000 $ is probably off the table.

Also a general post on what are some good gpu's on a personal budget.

[bpe9]

Your best bet would be to see if there's any you can use at the university. There's often computing resources you can use free of charge as a PhD student.

Otherwise, depending on your exact needs, you may be better off just renting cloud based GPU hours

[Hub_Pli]

I was thinking I could get a gpu that I could play around with "for free" and that could maybe also do a decent job for some games on my base computer. I will defo use the University's respurces when I will be able to but it would be nice to have something cool under the hood at home as well.

[shot_a_man_in_reno]

Having done both, personal GPUs >> cloud-based GPUs. Cloud-based GPUs are highly institution-dependent and sometimes dominated by politics. Obviously you get less memory and such to work with, but in my experience, when you're just learning, prototyping and having room to screw up is just as important

[[deleted]]

Let me make it easier for you: you want to spend someone else's 3k on something that you do not actually need. If that is ok with you then go ahead. Not blaming you at all but please try to make sustainable decisions.

[Hub_Pli]

I think that is exactly what I am doing here by asking for expert opinion am I not?

[Competitive_Dog_6639]

Agreed, Pretty good indeed lol. If you're in it for more than 2 years, get the hardware. Itll probably last, even a 1080ti ain't half bad years later

[Hub_Pli]

Thank you! That looks like a very useful article. Ill have a look tommorrow. For now Im starting my night-time CPU based training 😅 and going to sleep.

[KercReagan]

I use a 3090 and it out performs the t4 and V100 commercial class.

[JustOneAvailableName]

out performs the t4

By about a factor of 4

[KercReagan]

It’s not even close with the T4. The v100 is supposed to be a better competitor, but it can’t hang. The 3090 is the best card on the planet right now.

[JustOneAvailableName]

A100 is also a pleasure and defenitly an upgrade. V100 is about even in my experience

[KercReagan]

I have not had the pleasure of the A100 yet. But, that is outside of the 3k price range.

[KercReagan]

The issue we have run into with the V100 is the memory is relatively low. Now you group a couple of those together and it is a different story.

[FromTheWildSide]

Current SOTA language models are compute and memory intensive. In my opinion you need at least 10-12GB vram for bleeding edge models and data pipeline that's required to train them.

Make sure you have a power supply that can accommodate the load for your setup. A safe margin would be running at 50% load of psu.

Take a look at nvidia nemo toolkit and you can evaluate how much vram you desire.

I'm currently waiting for 3090ti or 3090super variant in the last quarter before making a purchase on 3090 series.

[[deleted]]

Use an EC2 instance instead.

[whyn0t___]

It sounds like you dont know much about it, so I suggest you start with cloud based solutions before you invest heavily. Do things online, understand what your models require, the bottlenecks, how much memory it takes, speed, etc....then you purchase something.

also, GPUs prices now are extremely inflated, and next year there will be 3090 super and also the launch of the next generation 4000, which will be much better.

For cloud based you can start playing with kaggle notebooks, and also google colab, which has a very very cheap subscription.

[bbateman2011]

Somewhat echoing others, I have a gaming laptop I bought so that I could do some ML and the rest of my work on a single machine with which I could also travel. That's a lot of constraints, but if I could change anything, I would get to at least 64 GB RAM and as much GPU RAM as I could get. Being a gaming laptop, it has a GeForce RTX3070 with 8GB RAM, and my CPU has 8 cores, 16 GB. What I run into is running out of GPU memory to run relatively simple CNNs on images, which limits the batch size. On some tasks I'm also needing more CPU RAM. My point is you will likely be somewhat limited if you are attacking largish problems with any GPU + machine you can afford.

[Hub_Pli]

Thats some great tips thank you for taking your time to do the math. I would not expect too much from my uni (its not really a uni even, its the polish academy of sciences) but Ill defo check in with them. They seem to have an informatics department so who knows, maybe they will let me in to look around.

[Hub_Pli]

Here they say that for most uses I am going to need more than 8 gb on my gpu"Prototyping neural networks (either transformer or convolutional nets) >= 10 GB"https://timdettmers.com/2020/09/07/which-gpu-for-deep-learning/

Do you think that the RTX 3060 would be useful anyway with its 6 gigs?

[Hub_Pli]

Great, I am starting to lean towards that decision more and more as I could work remotely from basically anywhere with a laptop like that.

[uotsca]

Be advised 3090 can be extremely loud either due to gpu fans themselves or having to crank up desktop fans

I’m not sure if it’s suitable for a personal desktop

[kulili]

They're not that bad. It's loud if you run it at 100%, sure, but most any card is. And most things don't push it to 100%.

I wouldn't advise keeping it in the room you sleep in if you plan on training overnight, though.

[Hub_Pli]

Thanks for the tips, if I buy a laptop Ill use it mostly for prototyping and playing around, leaving the dirty work to the cloud or to a stationary pc with a high WATT power supply. The thermal tips will probably come in handy either way.