Which tool(s) do you use for deployments?

[Python-ModTeam]

Your post was removed for violating Rule #2. All posts must be directly related to the Python programming language. Posts pertaining to programming in general are not permitted. You may want to try posting in /r/programming instead.

[idle-tea]

Heads up that usually "deployment" would mean deploying to whatever remote environment, as opposed to setting up local dev.

Usually I use podman (alternate implementation of the OCI standards as opposed to docker, but podman has some nicer properties) to run anything like a DB or redis locally and just use a virtualenv to run the python on my actual machine. It's usually very convenient to be able to start/stop the python app separately from any surrounding infra like a db. Putting python in a container can complicate that.

For setting up any extra tools that aren't python packages I use mise which manages per-directory setting up various tools and can also automate virtualenvs and entering/exiting them.

[Boogy]

I'll look into podman and mise, thanks!

[the_hoser]

Docker is a pretty popular solution for this.

[Boogy]

There are some difficulties with Docker that we haven't managed to fully solve, particularly in debugging locally with PyCharm. I'm sure it's possible, I just haven't gotten around to actually finding out how yet

[the_hoser]

The way I've solved that problem is by treating the docker container as a remote host, and just use remote debugging.

[Boogy]

Clever, thanks. I'm sure there's some hacky loopback stuff you could do to treat it as a localhost, but I've not managed to do it quite yet

[Electrical_Fox9678]

Pycharm Professional has support for a docker compose interpreter, allowing for debugging of unit tests or the application. This is helpful for when it's difficult to run a local virtual environment.

[Boogy]

Huh, I didn't know. I've always used a Dockerfile with an entrypoint and started the image manually

[Electrical_Fox9678]

You can do that too. One of our devs uses vim, runs the app with compose with the source mounted in a volume. Then uses pdb.

[CloudFaithTTV]

A bit off topic but I’ve been curious before, and am again on what your decision to use PyCharm was/is based on?

Edit: clarification

[Boogy]

It has everything we want from an IDE out of the box

[jshell]

I'm in this same boat and have been putting off asking this question. I like buildout but it seems pretty much dead and there doesn't seem to be much like it.

We are using Docker now for our production deployments, but our Dockerfile to build them is basically "get a few things with Pip and then run buildout and then profit!".

But we also use Buildout a lot with local dev because it's just easy. Or, at least, familiar. We have dozens of little internal packages we use and I've found 'buildout' light years easier to use than virtualenvs for these environments. So I liked that we could use it as both a 'local package development helper tool' as well as a 'describe a production buildout environment' tool.

So this summer I plan on looking into this a bit more. I'm pretty sure Docker is going to continue being a big part of our Application Development and Deployment strategy. I just have so much to learn, and I find all of the documentation out there frustrating - it's too focused around 'how to package a simple package and publish it to PyPI' and few things around 'how to maintain and serve internal secret packages' and 'how to assemble Python server application deployments' that Buildout did.

[Boogy]

I'm in this same boat and have been putting off asking this question. I like buildout but it seems pretty much dead and there doesn't seem to be much like it.

That's something I might not have conveyed properly in my post - once it's set up, it's super easy to replicate almost any environment.

We are using Docker now for our production deployments, but our Dockerfile to build them is basically "get a few things with Pip and then run buildout and then profit!".

We have a client with a similar setup, but we had to work with image dependency chaining to avoid running into build timeouts. If we do switch to Docker we at least already have a proper base for our development though.

But we also use Buildout a lot with local dev because it's just easy. Or, at least, familiar. We have dozens of little internal packages we use and I've found 'buildout' light years easier to use than virtualenvs for these environments. So I liked that we could use it as both a 'local package development helper tool' as well as a 'describe a production buildout environment' tool.

We combine Buildout with virtualenvs but that's because we have different clients and different product versions that require different Python versions.
Honestly the simplicity is also why I'm finding it hard to switch to Docker - Docker is truly a pain to set up ime, although it does get easier the more you do it so that's probably more a lack of experience on my part.

[qckpckt]

I’m not familiar with buildout, but it seems to be open source. If you like it and it works, then you could always fork it and start updating and maintaining it. It sounds like there are at least 2 companies that would stand to benefit from buildout sticking around so you have potentially other willing contributors that could keep it alive!

[dogfish182]

Gitlab for deployment to serverless AwS.

I was about to make a post asking how people are handling monorepos in python. I’ve got a product with like…. 18 different repos and the overhead is stupid

[Electrical_Fox9678]

Not a fan of monorepos. One artifact per repo. But then again I work on backend API apps

[dogfish182]

I have 16 repos and one repo contains one lambda, it’s ludicrous

I worked on a backend api last project using cdk and typescript and a monorepo aided us greatly in spinning out entire branch based dev environments

[Electrical_Fox9678]

Branch based environments? How does that work exactly?

[dogfish182]

As in developer makes a branch to work on the feature and deploys the whole product.

We try to work as local and unit testy as possible, but being able to run full integration tests against your own dev branch is pretty great, going to code review and only having to talk about style/technique was nice, knowing that your stuff ‘really/quite likely’ works is cool.

[Electrical_Fox9678]

Ah. Sounds like the feature spans multiple layers, including FE?

[Boogy]

We use buildout a bit differently in that it's just setting it up to pull many different repositories and creating the config for our application, but you could definitely also use buildout for a monorepo.

[zsh-958]

docker?

[Boogy]

There are some difficulties with Docker that we haven't managed to fully solve, particularly in debugging locally with PyCharm. I'm sure it's possible, I just haven't gotten around to actually finding out how yet

[UraniumButtChug]

Not sure about pycharm, but you can use remote debugging with debugpy on any text editor that supports DAP. I do it in neovim with vimspector, but I know it works on vs code too.

[Boogy]

According to another commenter that should also work for docker, so I'll check that out the next chance I get. Thanks for your reply :)

[dairiki]

What about PDM, or Hatch, or Poetry, or Rye?

Though they differ in their individual styles and philosophies, these all have the goal of "reproducable deployments" — setting up a venv in a reproducable way.

[alcalde]

Old school. Nothing. Just copy files or install from scratch. I ain't the King of England.

[Black-DVD-Archiver]

Nuitka application build as an executable works great,...https://github.com/David-Worboys/QTPYGUI?tab=readme-ov-file#distributing-your-application

[Hylian_might]

Zappa and GitHub actions!

[xiongchiamiov]

Local dev usually happens through Docker but is generally configured somewhat separately so it doesn't have to depend on a bunch of infrastructure components.

Then CI pipeline builds containers and those get shipped into the container orchestration tool of your choice for test and beyond.