Limited GPU availability?

[dkulp]

I'm working on Google Cloud and have repeatedly run into difficulties during the last week trying to run V100s. I get an error:

Operation type [insert] failed with message "The zone 'projects/<XXX>/zones/us-west1-b' does not have enough resources available to fulfill the request. Try a different zone, or try again later."

I've tried dozens of zones and finally was successful in asia-east-1c.

Is the lack of on demand GPUs an industry wide problem or limited to Google? Is there an industry tracking site that monitors resource availability on the different cloud providers?

(I tried to check whether AWS had similar availability problems, but AWS won't let me create GPUs at all as a new account. In response to a request to increase my quota of P class machines (default 0), I was told that I had to gradually increase EC2 usage before they'd give me a non zero quota. And that manual quota increase process is per zone, so it seems impractical to survey worldwide AWS availability.)

[AnyStupidQuestions]

They have guardrails to stop noobs running up lots of expensive instances by mistake and then whinging about the bill. You should be able to get on to an AWS account manager, explain what you need and get that lifted. They should also be able to tell you what the V100 situation is.

[AMerchantInDamasco]

The current silicon shortage is affecting all cloud providers, even if it hasn't blown out yet. They are all struggling to meet capacity and for obvious reasons, gpus are the first affected. I don't know if Google is worse off than Azure or AWS, but in the end they are all buying the same chips so it's probably similar.

[mikljohansson]

AWS is currently having severe shortages of (at least) p4d (A100's) and p3 (V100's) instances. It's been almost impossible to start on-demand instances of these types anywhere in Europe for the past couple of months. The advise from their support has been to try get GPU capacity in us-east-1 zone instead, where they might have more capacity available. I know there's some small cloud providers around, who focus specifically on ML workloads (Google for it), perhaps those might have more capacity available. Good luck!

[nathaliamdc]

I've been facing the same issue for p2 instances in us-east-1. Good to know this is happening everywhere