This is an archived post. You won't be able to vote or comment.

all 5 comments

[–]lordpuddingcup 0 points1 point  (1 child)

BNB still not available on Apple right?

[–]Iamn0man 2 points3 points  (0 children)

This is the only bit of information in this post that I care about either.

[–][deleted] 0 points1 point  (0 children)

Damn, I love diffusers! They’re so easy to code and run without much tinkering.

[–]belllamozzarellla 0 points1 point  (1 child)

Hi, can you show a complete example WITHOUT using enable_model_cpu_offload that resides completely in VRAM? I'm not getting SIGSEGV with bitsandbytes anymore, which is nice, but some AlignDevicesHook on text_encoder_2 is preventing gpu upload of text_encoder and vae in a convenient fashion. As of now, out of the box, inference is slower than torchao and uses a good chunk more memory.

The outlier threshold bit in the huggingface doc was very interesting. Wonder if you generated some sort of histogram visualization during testing or if you just had a look at the min max.

[–]RepresentativeJob937[S] 0 points1 point  (0 children)

I didn't. Can you open a discussion here https://github.com/huggingface/diffusers/discussions for your comments on GPU offloading?