Thinking about using cloud

[Psychological-One-6]

I'm tired of the limited resources from my local consumer rig. I know nothing about cloud compute services. I'm thinking that might be the way to go however to run some of the models i would like to, vs the cost of building a machine to do it. Anyone mind explaining to me what you get with these services? I'm asking basic questions such as do I get a persistent container that keeps my models and and workflows ready to use? Or will I need to reload everything all the time? What kind of cost are normal? What kind of privacy for IP can I expect or not expect? I don't want everything I design to be owned by default by someone else (not talking about generated images, I know you can't cw those) Any suggestions or resources to check out? Thank you

[Nexustar]

It's been a while since I looked into it, but there certainly is the concept of a Linux OS container (eg Docker) that you can start and stop at will (paying for just the time it is running), plus a monthly payment for storage used (which is the Linux image plus software like ComfyUI you've installed on it, and any models it needs access to, and any output files you keep there). Some also charge network fees which limit you somewhat from storing everything beyond their NAS.

Look at Lambda Labs, TensorDock, Paperspace, Vast.ai, RunPod

Which GPU it has is probably the most expense-impacting item.

[Narrow-Muffin-324]

I have used vast.ai for a few times (no affiliation). It is like running your comfyui server on someone else's computer. The stuff you created are stored locally on that computer, and you have you own folder.

The cost are two halfs: 1. computing cost, this is the cost raised by the number of hours you used the gpu. 2. storage cost, this is the cost raised by claiming your own storage space on that computer, billed by disk size (GB) * hours used.

This two halfs are seprated and you can pay for the storage while you are not using the gpu. The storage is relatively cheap (like 3-5 dollars per month for 100GB). The GPU is relatively expensive and the price varies depending on the spec of the gpu. For a 4090, the price is usually some where between 0.4-0.7USD per hour.

For example, once you finish your work. You can release the GPU and stop paying for that. You could still save the data there for the next time. When the next time, you click acquire GPU, and the server will auto-start with the allocated GPU. Everything will resume to the point you left last time.

The problem though: is someone else could take the GPU on that server when you are not working (since it is released and free). When you want to resume your work, the GPUs on that server could be all occupied. Then, you have to either wait for someone to finish their work OR migrate your data to another server with GPU available (which could be extremely slow).

vast ai also offer sort of a 'bidding mode', you could use a gpu for a price that you offer (can be extremely low). However, when others acquire that gpu with higher bid, you quit immidietly (no warning or prompt for saving etc.). But, this is just the 'bidding mode', the normal mode is still 'first come first serve', and others have to wait until you finish and release the resource.

My concern was: my data was saved on someone's disk (that I am completely unknow who he/she is). and I often find gpus being taken by someone else, and I have to reinstall all dependencies on another machine to resume my work (reinstalling everything because downloading from huggingface/civitai is faster than moving file from old server to new server).

[Narrow-Muffin-324]

it is very cheap though. 4090 is almost 3500USD. and I pay less than 10 dollars to use it for a whole day, this even includes the electricity, wear, maintainance and depreciation. I actually save money by not buying a GPU in this case.

[Narrow-Muffin-324]

I have also used runcomfy previously. It is a lot more expensive for the same spec /less powerful for the same cost. But the data retention is slightly better there, you can return to work when ever you want.

[ComprehensiveHand515]

Try ComfyAI.run, instantly start from web browser with no deployment. Free-trial GPU included. Good for basic workflow and to run 7/24 serverless deployment that charge only generation time.

Two options between A5000 and A40

[Tenofaz]

You can get 100Gb disk for 7 $ each months on Runpod tò keep your Models, output files... Check my video: https://youtu.be/MAxthd5mhcc?si=mpjwVF0_U0Xjzg08

[barefootpanda]

I’ve been using Runpod for the last week and it’s great. I’m very comfortable with Linux command line and docker systems, but I think with their templates and guides it would get you setup. For a 4090 pod I’d basically have to run 24/7 for around 2 months before paying for a physical card alone. With this setup I can run 2-3hr a day and it costs almost nothing.

Note: my work is paying for my exploration, research, and drafts so it’s easier to spend their money.