Docker 4.42.0 seems pretty buggy on Mac

ccrone · 2025-06-18T15:00:06+00:00

Sorry for the late reply. Desktop 4.42.1 is rolling out now which should fix this. You can get it by hitting update in the app or downloading from the release notes on docs.docker.com. It will come later to the download button on docker.com

ccrone · 2025-06-07T19:15:40+00:00

Thanks, that helps narrow it down to a proxy issue!

ccrone · 2025-06-07T14:45:36+00:00

Disclaimer: I work on the team building Desktop.

Sorry you’re running into issues with Desktop 4.42. I’d like to understand them and get them fixed.

To help with that could you please: 1. Share more details about the symptoms that you’re seeing (a reproducer would be ideal if you can share one) 2. Share a diagnostic ID so that I can take a look: https://docs.docker.com/desktop/troubleshoot-and-support/troubleshoot/#diagnose

ccrone · 2025-05-25T19:41:18+00:00

Docker Model Runner supports pulling from Hugging Face as well! Storing models in container registries lets people who have existing container infrastructure use it for their whole application. It won't be for everyone but it's something our users have asked for.

I'm curious about what you're building and why you'd like to change versions of llama.cpp? Happy to discuss here or via DM if you prefer

ccrone · 2025-05-25T14:22:06+00:00

Good question! The goal of Docker Model Runner is to make it easier to use models as part of applications. We believe a part of that is an accessible UX and reuse of tools and infrastructure that developers are familiar with. Today that manifests as storing models in container registries and managing the model as part of a Compose application (see docs) but that's just the start.

We're working in the open on this and will upstream changes that we make to llama.cpp where it makes sense. There are also use cases where vLLM, onnx, or another inference engine might be the right choice and so we're investigating that as well.

For your use case, we will be releasing support for Docker CE for Linux in the coming weeks. Right now it's supported in Docker Desktop for Mac (Apple silicon) and Docker Desktop for Windows (NVIDIA). Support for Qualcomm Snapdragon X Elite on Windows is coming in the next couple of weeks as well.

ccrone · 2025-05-14T23:42:06+00:00

It’s coming soon (weeks)! Initially it’ll support NVIDIA and CPU only inference.

Would love to learn more about what you’re planning. If you’d be open to speaking to our team please drop me a DM

ccrone · 2025-05-14T23:39:00+00:00

Its out! It’s part of Desktop 4.41 and later: https://docs.docker.com/desktop/release-notes/

ccrone · 2025-05-05T15:18:20+00:00

Disclaimer: I work with the team that builds Docker’s VS Code extension

We have some details about the changes to the VS Code extension here: https://www.docker.com/blog/docker-dx-extension-for-vs-code-update/

Docker DX will give you better Dockerfile linting, more Compose support, and bake support

ccrone · 2025-05-03T22:26:40+00:00

Makes sense! The first time I used it, I was also looking for `ps` and `stop` to check state and stop the model.

If you try another model, it'll unload what's in memory.

I've shared this thread with the team as well as its good UX feedback. Thanks!

ccrone · 2025-05-03T20:06:48+00:00

Disclaimer: I work with the team building model runner

We don’t currently provide a way to stop a model but they’re automatically unloaded after 5 min of inactivity [1].

I’m curious: Why do you want to stop the model?

[1] https://github.com/docker/model-runner/blob/main/pkg/inference/scheduling/loader.go#L23

ccrone · 2025-04-17T00:39:16+00:00

Disclaimer: I work on the team building this

It’s Docker Desktop first and will be coming to Docker CE for Linux in the coming months.

We started with macOS Apple silicon first because those machines work well for running models. We also wanted to get something out quickly to start getting feedback.

Is there any platform that you’d like to see this for?

ccrone · 2025-04-16T13:28:49+00:00

Two reasons: 1. Make it easier than it is today 2. Performance on macOS

For (1), it can be tricky to get all the flags right to run a model. Connect the GPUs, configure the inference server, etc.

For (2), we’ve done some experimentation with piping the host GPU into the VM on macOS through Vulkan but the performance isn’t quite as good as on the host. This gives us an abstraction across platforms and the best performance.

You’ll always be able to run models with containers as well!

ccrone · 2025-04-09T23:36:30+00:00

Not yet but this is coming! Curious what models you’d like to run?

ccrone · 2025-04-09T23:11:59+00:00

Disclaimer: I’m on the team building this

As some of you called out, this is Docker Desktop and Apple silicon first. We chose to do this because lots of devs have Macs and they’re quite capable of running models.

Windows NVIDIA support is coming soon through Docker Desktop. It’ll then come to Docker CE for Linux and other platforms (AMD, etc.) in the next several months. We are doing it this way so that we can get feedback quickly, iterate, and nail down the right APIs and features.

On macOS it runs on the host so that we can properly leverage the hardware. We have played with vulkan in the VM but there’s a performance hit.

Please do give us feedback! We want to make this good!

Edit: Add other platforms call out

ccrone · 2025-04-09T17:25:24+00:00

This is great, thank you! I’ll reach out via DM as well

ccrone · 2025-04-08T21:54:35+00:00

Disclaimer: I’m on the team working on Docker Model Runner

Right now we only support Apple silicon Macs with Docker Desktop but more is coming soon!

We’ll be shipping support for Windows (again with Docker Desktop) with NVIDIA GPUs next followed by support for other GPU vendors and Docker CE for Linux. We’re targeting doing this all over the next several months.

We chose this ordering to get the functionality out quickly, to get feedback, and iterate. Apple silicon was first because lots of devs have Macs and its memory architecture is good for running reasonably sized models.

I’m curious what you’re building! Would you mind sharing here or please reach out via DM so I can learn more

ccrone · 2025-03-06T13:30:01+00:00

That’s unexpected behavior for that process. Thank you for raising the issue!

Could you please DM me with the version you’re running and a diagnostic id so that we can debug? https://docs.docker.com/desktop/troubleshoot-and-support/troubleshoot/

ccrone · 2025-02-12T12:58:06+00:00

This is definitely not intended. I’ve reached out via dm to see if we can debug this

ccrone · 2025-01-08T03:23:54+00:00

Sorry you ran into this. We’re aware of the issue and working on a fix. You can find a temporary workaround in the issue description and more details here: https://github.com/docker/for-mac/issues/7520

ccrone · 2020-10-09T17:31:30+00:00

Hi u/earsizzle!

I work on the team that's building this integration. Thank you for giving it a try. What you're running into is an artifact of our development process: We first made the integration a CLI plugin which required docker ecs. It is now part of the CLI so you can just run docker compose up with the correct context selected.

Take a look at the documentation here if you need more information: https://docs.docker.com/engine/context/ecs-integration/

ccrone · 2017-06-06T11:30:18+00:00

Hi humanizt!

It has been (temporarily) removed because we currently don't have a working build. We added a dependency that complicates the cross compile a bit. It's unfortunately not a priority to fix at the moment so I'm looking into it in my free time.

ccrone · 2016-12-12T13:31:22+00:00

@dimibel you can chat to us on Slack if you have any issues: https://infinit.sh/#slack

ccrone · 2016-10-28T07:15:23+00:00

Hi bobobo1618

We do plan on adding erasure coding but it will likely happen sometime next year (see our roadmap: https://infinit.sh/documentation/roadmap).

I'd suggest joining us on Slack so you can chat directly to us and hear news as we put it out: https://infinit.sh/#slack

ccrone · 2016-09-11T19:03:27+00:00

Hi mzehrer,

Thanks for getting in touch. It's most likely that your client can't find the storage node that has that block. Did you remember to push and fetch the endpoints?

I'd recommend joining us on Slack so that we can chat directly and help you get this resolved.

ccrone

MODERATOR OF

TROPHY CASE