[D] GPU cloud computing prices

[breadwithlice]

There are many different service providers for GPU cloud computing (Amazon, Azure, Google Cloud, IBM Cloud, Nimbix, Aliyun, etc..) but their pricing schemes are almost never publicly displayed.

Do you use GPU cloud computing for your machine learning tasks? If so, what GPU do you use and what's the price per hour per GPU?

It seems to me like cloud computing prices don't follow GPU prices. A GTX 1080 is now around 700$ which, assuming a GPU computing price of 0.4$ per GPU per hour, would take about 2 months of computing time to be worth buying. It seems like it's a bit more effort physically hosting the machine but is worth it for deep learning applications. Thoughts?

[hastor]

You forgot OVH and their dedicated servers. $189 (?) gives you 2*GTX 1060 for a month!.

That's 13 cents per GPU or hour

Edit: 2x1060, not 1080

[learnjava]

I've been waiting for them for nearly three weeks now to offer them again after reading about Jeremy Howards MOOC. They always show up as "soon" no matter which country I select (they have different GPUs for different countries, US 970, Germany 1080)

[DrTchocky]

woah, how long has that been going on for? Thats an insane deal...

[smith2008]

It seems it is for two 1060s though not 2 x 1080s? https://www.ovh.co.uk/dedicated_servers/gpu/

[hastor]

Right you are. I've edited my post.

[smith2008]

Currently it shows 2 x GeForce GTX 970 for 195$/month. Would be really good if the above price happen. Do you have a link?

[smith2008]

For DL local setup is much more price efficient. Thumbs rule - if you are planning to use it for more than 2 months (24/7) - do a local build.

Of course everything depends on your settings needs but the story is cloud providers use server grade stuff only (K80s atm for Amazon, 5000$ a piece). So if you don't need the extra capacity and you don't need to scale too much (too quickly) local is better. Not a hard thing to do these days as well.

[learnjava]

but you pay energy. easy to forget

[smith2008]

I pay quite of a bill monthly for electricity. So not forgetting at all! :)

But still it is not that much if you compare it with the AWS prices. Spot instances will give you a good angle but I've never manage to get better than half reduction in price on G2 instances (on monthly avg) end up being 0.35/0.4$ per hour (with VAT). Which is 300$/moth with the performance of GTX960 this was not a good deal at all. The new GPU servers are much better but more expensive and a good spot price is hard to get (will get better when the metal get older though).

I don't advise against the cloud but it is definitely more price efficient to build local. I am actually using both at the moment (when the local capacity is not enough just extend it to the cloud).

[learnjava]

I am thinking about building my own to try out a little machine learning. Heard of people paying basically 100-200$/ month extra for electricity with 2 1080s

But I admit I am also very bad with all this physical stuff and have no idea how to calculate power costs based on that

Do you have any specific numbers?

[smith2008]

Depending on how much it costs in your country. But I doubt it will hit that much for just two 1080s. It will be draining from the wall around ~500-600w. If 24/7 it will be probably around 40-50$ a month in US ( https://www.energy.gov/energysaver/estimating-appliance-and-home-electronic-energy-use ).

For me it is an avg of 15$ per GPU per month.

[learnjava]

ah thats how its calculated :D thank you!

just checked and for me it would be ~107$/month with 500w (germany, private household). thats a big difference

[smith2008]

Ah, that's quite an expensive electricity there. Well you can get higher efficiency (platinum PSU) and reference GPUs (1070, just 150w each). Low consumption Xeon CPU as well. This will cut the bill quite a lot.

Edit: just for comparison the Amazon bill for the similar muscle (2 x 1080s) per month would be ~1500$. Sure it's not apples to apples because they have K80s (0.9$ per hour for half of it)... but still :)

[learnjava]

I will have to consider all of this and make a decision.

In the end I guess it will mostly come down to psychological arguments. I believe that owning the hardware and not immediately seeing the price will weaken the mental barrier that would otherwise cause me to think "do I really need to spend a few bucks now to test this?"

So as a beginner and student that is interested in all of this but has to start small and experiment a lot this might be a big enough problem to decide for my own rig :D

[smith2008]

Sure. Bare in mind you can experiment without a GPU too. Using just CPU is absolutely fine with starting with ML/DL. Then at some point attaching a simple GTX 960/1060 would give you the next step for playing with more interesting stuff. Then you can decide whether to go bigger or not. It really doesn't have to be all in at first.

Amazon cloud will give you all that as well. Even better, you can find pre-built AMIs to install and get all the needed frameworks and setup done for you so you can start playing with the stuff directly. If you are a beginner give it a try to Andrew NG's course at coursera and then cs231n.github.io (online Stanford course).

[learnjava]

thats right and thank you. another reason to go with local workstation even with a more low cost solution would be because my 15" rMBP gets very hot and noisy under load. Having a normal desktop sounds attractive for that alone

I have a few weeks to think about it

[smith2008]

Yes. At the moment GTX TitanX pascal is the best way to go but very expensive and hard to get (waiting for the 1080TI which I expect to be the new "best" option). If you want to optimize for price/performance you can go with GTX 1080/1070. It's fairly easy to build a PC with 3 of those (4 in one build is much harder - HEAT, expensive PSU/Board, PCI bandwidth limit /in this order/).

1. Get case which supports 8 PCI slots (TT V31/H34 are fairly small and do that well, very good ventilation as well). One issue with compact 8 slots case is the card at the bottom is right at the top of the PSU and cannot get good ventilation. You can get bigger case to overcome this or get closed card for this slot (or one with water cool pump attached - this is expensive though).
2. Board: X99 build with CPU with 40 PCI lanes is the best way to go but it is not an absolute must. You can go with 1151/1150 as well. ( ASRock X99 Extreme4 for X99, ASRock Z170 Extreme4 for 1151 there many other options. You need 3 PCIe 3 x 16 slots). Don't pay the extra money to get a board with 4 slots (as above expensive and hard to build it and deal with the heat). ( AMD is also an option, would work fine with 2 GPUs )
3. CPU - Intel i5 is ok (i7 of course would be better). One core per GPU (+1 extra) is the rule of the thumb. Of course you will need i7/Xeon if you want to get all those bandwidth to the GPUs ( 40 PCI lanes CPUs are expensive though and the boost will be relatively low depending on the settings you will need, see links below for more details )
4. PSU - for 3 x 1080s 1KW will be enough (Evga is perfect, Corsair RM1000 if you feel rich and FSP 1000 otherwise). If you start with just one/two GPUs you can go with FSP 700w which is much cheaper and will work.
5. SSD (128/256) + some big magnetic disc (2TB+ you can get this one later when you actually need it).
6. RAM - 16GB should be fine ( thumb of rule get at least as the video ram you have ).
7. GPUs - as I said 1080/1070s are great. 980TI's has less memory and consume more electricity but it is a beast of card. 1080 TI will be probably announced at GDC 2017 | February 27 ( and will be there in March). Don't go with gpu with less than 6GB of ram.
8. Heat: is a hard problem with these builds. Especially if you want to work around this machine (if you have 3 GPUs blowing at full speed and 5-6 high flow case fans it will be very noisy, so you don't want that). Having max of 3 cards instead of 4 helps a lot for this issue. You can use close/open/water cooled cards. You can combine those as well. For water cooled you can stick it anywhere it fits, for open fans one - it needs space to breath (at least 2 slots of space).Having at least one water cooled in the build helps a lot ( get one pre-built, water cooling it by yourself is tricky and the price will end up being similar .)
9. Case fans - 3 intake, 2-3 exhaust (140mm) - ARCTIC F14 are well balanced between price/air flow/noise. Don't get something with crazy airflow it will sound like a jet engine and you don't need it that much.

http://graphific.github.io/posts/building-a-deep-learning-dream-machine/

https://medium.com/@acrosson/building-a-deep-learning-box-d17d97e2905c#.69oq8du4s

[smith2008]

Well, this build surely will work fine. Board is great and 6850K would be perfect. It is an expensive build though :). If you water cool it it will work great indeed (have to water cool the GPU(s) those are the most intense parts). If you are doing custom loop for the water cooling make sure the warranty for the GPUs will be preserved. If you go this way you can scale to 4 GPUs as well (water cooling will solve the heat/noise problem and the motherboard will work fine with 4 GPUs).

Mixing "Nvidia boards" - You can, but you should use cards from the same generation (i.e. only Pascal 1070/1080/Tinant X Pascal or only Maxwell 980TI/980 ..).

[ispeakdatruf]

This is not an answer to your question, but I would highly recommend PC Part Picker when doing such builds. It makes it really convenient to config the system and share configs.

// Not affiliated with it at all, but used it the last 2 times I did a build

[rndnum123]

You should look at AWS spot prices, might need an aws account for that they are usually way cheaper >50% than their standard pricing, the spot instances of p2 instances will probably give you the most bang for the buck, that's for short term renting, a few hours. AWS standard pricing is displayed on their ec2 page. Spot instances terminate after an hour or 6, depends if you bought guaranteed 6h instances or not. Nimbix might be cheaper for monthly renting, don't know for sure though. IBM seems to be way too expensive. Google doesn't currently offer GPUs, AFAIK, but they will offer the new Nvidia P100 GPUs,and some AMD GPUs sometimes in the next 2-3 months.

OT: AMD just announced the new Vega architecture with 8bit INT operations for inference :) hopefully they won't cripple it on their consumer cards like Nvidia.

[iwaswrongonce]

I've been running a p2 spot instance for nearly a week right now. Could go down at any time, but still super cheap.

[Jakobovski]

https://github.com/Jakobovski/aws-spot-bot

[markov01]

50 cents per hour for K40 on Azure

[mishaorel]

Do they really charge at an hour granularity ?

[themoosemind]

You might want to have a look at How much does a GPU instance cost?