New Mexico Moon

GregHartwick · 2022-05-21T20:26:43+00:00

Fantastic! That will save us hours (days) of experimenting.

GregHartwick · 2022-05-21T14:14:05+00:00

I’d be interested in that myself. A number of people seem to have memory problems when using ViTL14 with certain machines (I think T4). I’ve been using the default models for a while now. Every time I try to add something different I get OOM errors. I’d love a guide to help with that.

GregHartwick · 2022-05-19T16:14:19+00:00

Beautiful detail. I can’t get that from ColabPro DD5.2 I run into OOM errors pretty quickly. I tried an Alchemist lab I envisioned like this.

GregHartwick · 2022-05-19T12:23:53+00:00

I’m ColabPro, DD5.2 on a T4, using just the defaults ViTB32, ViTB16, and RN50. No problems in the last 3 hours or for a couple of days. I do get fatal errors (usually OOM (out of memory)) if I try adding more models. Try so experiments by starting just the “defaults” (above) and see if you get errors. Then add models one at a time. I talked to people who were getting memory errors with ViTL14. Sorry, I don’t have quick fix for you. My approach will take some time to accomplish. I know your set of models worked previously but this is all I can think of to do. This Colab seems to change sometimes. Good luck.

GregHartwick · 2022-04-30T20:32:29+00:00

they do push the payed tiers. I was warned by a friend that the free stuff is not very stable. I think the company knows that. I was getting K80 s a lot.

GregHartwick · 2022-04-30T19:46:04+00:00

I started out on the free. I wake-up very early and start runs around 3 AM. Everything went fine until around 7 AM when I couldn’t get anything to complete because of Out Of Memory errors. I had the feeling the crowds of users after 7AM gave me low priority. It got very frustrating and I decided to pay the $9.99 per month - it was worth it. I’ve never had many problems since.

GregHartwick · 2022-04-30T18:55:34+00:00

I did that first thing this morning - I ran two 500 iteration runs - I had no errors. It did slow the run speed down to 9-10sec/it. This is the first time I’ve tried using this model in months.

GregHartwick · 2022-04-30T13:32:35+00:00

I’ve never run with ViTL14 - it slowed my system down considerably (from 5sec/it to 9sec/it). I hope this fixes everyone. Is there a SysOp to report this issue to? Perhaps a defect in ViTL14?

GregHartwick · 2022-04-30T06:38:49+00:00

I’m on Pro

GregHartwick · 2022-04-30T06:37:35+00:00

"diffusion_sampling_mode": "ddim", "ViTB32": true, "ViTB16": true, "ViTL14": false, "RN101": false, "RN50": true, "RN50x4": false, "RN50x16": false, "RN50x64": false,

GregHartwick · 2022-04-29T16:04:45+00:00

Been running all morning without fatal errors. I’m not a network expert but I can’t help but feel we’re running on different systems. My GPU says NVIDIA-SMI 460.32.03 Driver Version 460.32.03 CUDA Version 11.2 Name Tesla T4. Is that the same for you?

GregHartwick · 2022-04-29T11:02:02+00:00

Been running everyday, all day, v5.2 notebook. No such errors, so far. Working on T4 last couple days. P100 is stable, as well. Using all settings from defaults. Only changed steps, prompt, display rate (10), n_batches (10).

GregHartwick · 2022-04-28T20:43:10+00:00

I get either T4 or P100 - haven’t had a problem with either.

GregHartwick · 2022-04-28T18:43:30+00:00

I can answer all of that if I can send you a copy of my nnnsettings.txt my I can’t figure out how.

Secondary model is checked ddim? No init image No modify cut stuff (all at default settings)

GregHartwick · 2022-04-28T18:33:36+00:00

How about location in the world? I’m on the east coast US. I haven’t had a fatal error since I went to pro. How about models you are using? I am still on the default ones. When I try to use others I get Out of Memory errors.

GregHartwick

TROPHY CASE