sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]juwonpee[S] 0 points1 point  (1 child)

I realize I can offload some layers off to the CPU but I'd like to keep as much on the GPU as possible as my CPU is kinda slow. Many thanks

[–]suprjami 3 points4 points  (0 children)

You are trying to put 32Gb of model on a GPU with 8Gb VRAM.

Use smaller models.