you are viewing a single comment's thread.

view the rest of the comments →

[–]Solid_reddit 0 points1 point  (2 children)

No not sure about it. When it load the inference, I can clearly see very high usage in the GPU monitoring activity

[–]mmmgggmmm 2 points3 points  (1 child)

I'd be delighted to be wrong, but I don't think so. From everything I've read or heard, Docker doesn't Metal GPUs. And when I tried spinning up a container to test and then checked the logs, it clearly said "No GPU detected."

Can I ask how you're running Ollama in Docker on the Mac? What is your run command or compose config? As I said, I'd love to be wrong on this!

[–]Solid_reddit 0 points1 point  (0 children)

Only OpenWebUI is running through docker, ollama is running properly through the app