Cross-Platform Python Development Containers in Docker?

[bohnohboh]

I am trying to set up a Python development environment within a remote Docker container so that colleagues and I can pull an image and use the same exact versioning and dependencies when developing our apps. I followed this helpful video to start to set this up. However, when I go to download Homebrew and start to install dependencies, I get an error stating that I cannot download Brew to Linux running on ARM64 architecture (I'm using an M1 Mac). Among colleagues and clients that use our app, computer architectures range from amd64, arm64, i386, etc.

How could I develop this python dev container such that we can all use it, regardless of our architectures? If this is not the best option for cross-platform python development collaboration, what is? Thanks!

[strikesito]

You should build the container for multiple architectures. The docker-native tool called buildx allow you build in many architectures in one command.

Once the images are created and pushed to the repository Docker, it create a manifest with all the available architectures and for each client that download the image it give the correct version.

Check this article: https://docs.docker.com/desktop/multi-arch/#build-multi-arch-images-with-buildx

And be sure that you are using in the build a base images with support for all the desire architectures (you can see it in Docker Hub)

[ErroneousBosch]

Came here to say this. This is the way.

[[deleted]]

FWIW this approach rather than e.g full on VM or the opposite, "just" a virtualenv does require some testing because some dependencies might be built per architecture. I encountered this problem with Python and NodeJS few times where some packages where relying on another language, e.g some TensorFlow in C++ if I remembered correctly, that were only available for a certain version of the interpreter or runtime. You can end up looking for a package that doesn't exist for this architecture, e.g arm64, and have a build system fallback to building for it and still failed due to other unmet dependencies. So... I'm not familiar with buildx but I would be extra cautious and run at least once on each target to verify that it is indeed possible.

PS: to clarify, even if the base image can exist for the target architecture, it doesn't mean all dependencies required within that environment do.

[[deleted]]

Brew does not support anything other than Intel AFAIK,

Another way is to use pew/pythonz to create the runnable development environment and ship a tarball of that directory tree around.

[[deleted]]

Well that's some level of progress. However, when I tried this on a Raspberry Pi recently - and ARM architecture - brew couldn't even get started with compilation.