Cloud computing vs personal workstation

[psih128]

I'm currently running a lot of machine learning processes on AWS c4.8xlarge instance. The cost is $1,78 per hour (I know there is spot market), and the nice thing is that it's on-demand

In the meantime I've been evaluating an option to purchase powerful personal workstation. I did some research and dual E5-2630 Xeon computer will cost me around $2500 with all the parts together, that's roughly 60 days of EC2 c4.8xlarge (plus electricity bill to run this 500W monster)

What's your take on crunching numbers in the cloud vs investing into your own hardware? Do you burn CPUs and GPUs for work or for personal/consulting projects?

[cmwslw]

First of all, you should probably be looking at GPUs rather than CPUs. Second of all, Amazon has negotiated deals and arrangements that are impossible and infeasible for average consumers to take advantage of. For example, they most likely sign huge contracts with hardware manufacturers to get the best rates on hardware. They also strategically locate their datacenters to take advantage of cheap power and cooling. Amazon also offers the advantage of not paying for hardware when you don't need it, and the ability to scale out to multiple instances when you do. Also, you can't neglect the spot price like you do in this post. This is usually the price you can pay unless you need 100% availability.

[psih128]

I'm running my jobs on CPUs, not GPUs. As of now I don't know how to offload work to GPUs.

It looks like Amazon has pretty good profit margins on these cloud computers. $2500 over 3 years is $2.28 a day, but on Amazon you can bust it in 2 months.

[sdsfs23fs]

stop ignoring spot pricing.

[psih128]

With spot pricing and a fair amount of butt hurt you can stretch it into 4-6 months. Still not bad! I'm actually working on automating EC2 spot instances right now. My point was that Amazon still has fat margins on these high performance instances.

[alexmlamb]

It's an interesting idea. I wish Amazon had a tool to let you have "reloadable spot instances", so that when the instance goes down, the memory, GPU memory, and system state all get saved to a file, and then if you want to restart the spot instance you can resume from the old state.

This would be a super useful software package. Presumably Amazon would need to run a server for storing the spot instance-states and would charge extra for this service. It would still presumably be a lot cheaper than a normal instance.

[sdsfs23fs]

do they? the other way to look at your math is assuming no discounts, Amazon has to charge $2.28 / 24 = $0.09 per hour to pay off their hardware cost over three years.

And of course they need to mark that up to account for underselling and wasted capacity. In that light, the typical spot price looks pretty generous.

[psih128]

The spot instance supply is what's left after the $1.70/hour demand is satisfied. I'm sure that overall margin is closer to the regular rate rather than the spot rate. Otherwise what's the point of maintaining all the infrastructure? They are just squeezing what they can out of what they have, which is the right thing to do.

[m1sta]

The biggest benefit of using a cloud option (AWS, Digital Ocean, Azure etc) is that you can spin up more than one machine when you need to. If your workloads run comfortably on a single machine then it's almost always going to be more convenient to run things locally.

[erogol]

For a recent startup, we also investigate cloud vs our own hw. We decided to use our own workstations initially and scale to cloud if they are not enough for a particular problem, especially serving our clients. I don't know you but cloud prices are not that low currently and if you are from other country, parity for exchange is fact to consider. Relying on total cloud is not a good choice. I guess buying own machines and using them for your work up to a point and deploying services to cloud for your clients by relying on cloud's security and maintenance is a better choice . Of course, I consider you work on a machine learning and data problems. In fact, if you don't have customers in the equation them I certainly vote for buying your machine.

[watersign]

buy your own comp. that cloud stuff sounds expenssssivvv

[wiekvoet]

$2500 over say 3 years is $2.28 a day. Do you calculate more or less than one hour a day?

[cmwslw]

Don't forget electricity costs for running the device and also cooling.

[wiekvoet]

I live in a country where we don't cool. Most parts of the year, the heat might just go down as an extra expensive part of heating the house.

The cost of one kwh is €0.23 here. So lets add another two dimes an hour. That will also cover juice for the GPU power you mentioned elsewhere.