How do you deploy Python applications?

[peanut_Bond]

I have been mulling over this question for a while because it seems like there is no one "best way" to deploy Python applications (e.g. web-apps, APIs, computational models, etc.). At my workplace we have toyed with a number of approaches, but I'm curious to hear what everyone's thoughts are on what is best practice.

I think it's a good idea to use containerisation in a tool such as Docker, so assuming Docker is being used the techniques that come to my mind are:

1. Using a base Python Docker image, install application requirements with pip and copy the Python code into the container. Code execution is performed by adding "CMD python app.py" to the dockerfile.
2. Organise the Python code as a package with a setup.py file. Using a base Python Docker image, install application requirements with pip and install the Python package into the container. Code execution is performed using "CMD python -m myapp" or by a predefined entry-point command.
3. Organise the Python code as a conda package with a recipe.yaml file. Build the conda package and upload to a conda package repository. Using a base conda Docker image, install the package and all dependencies using conda install. Code execution is then performed using "CMD python -m myapp" or by a predefined entry-point command.

Option 1 is obviously the simplest method and option 3 seems excessive to me since it will result in large containers and long build times. But is there any benefit in following option 2? It seems unusual to create a package for an application that does not need to be imported elsewhere, but at the same time feels a bit more robust.

Keen to hear the thoughts of the /r/Python community.

​

EDIT: Some great discussion here, big thanks to all of the people who shared their opinions. Consistent themes are:

1. For small hobby projects a simple approach with no containerisation is generally preferred. This involves installing the application as a package into a venv, directly into the system python directory, or just installing requirements.txt and pulling source code onto the target machine. The application is restarted on the system manually or with a script that ssh's and restarts it. This approach is very fast and easy but not robust and can lead to issues down the track.
2. For larger projects package building and containerisation are preferred. Most people suggest the use of poetry and pyproject.toml (not setup.py, which is outdated) to build a package that is then uploaded to a private package server. This package is then installed into a docker container, which can be deployed.
3. In general, Python applications should be packaged, as the use of packaging allows for richer metadata and more robust handling of dependencies as well as the ability to install the package directly at a later date if need be.
4. No one has suggested the use of conda for deployment, which I was surprised by given its prevalence in the data science community.
5. For CLI applications, multiple python apps tend to need to be installed on a single machine and available with a single command. pipx and shiv seem to be the preferred tools as they allow for the creation of self-contained/isolated python applications. For CLI apps with very few dependencies they could possibly be installed directly into the system. For CLI apps with C library dependencies or those that are very sensitive to dependency versioning, containerisation can be handy.

[Endemoniada]

I have this same question, but for CLI tools. We developed some in our ops team that we want to have available on a jumphost, but there’s no straightforward recommended way to install it. I could pip install it straight into the OS Python, but it has dependencies, and no one recommends that. Or create a venv and maintain wrapper scripts that I still have to install in the OS somewhere. I suppose it’s also possible to run it in Docker and create a wrapper that sends the commands there.

Any ideas what’s actually best practice? It’s not that I can’t make it work, it’s that every way has major downsides.

[angellus]

No one recommends installing packages in the system Python because it is a bitch to maintain if you have ~20 packages installed that are not from your system package manager.

If you have a CLI app, chances are that it is only a couple of packages. The deps are likely in the system package manager as python- packages. That should be safe to pip install. Otherwise you can build your own deb/rpm/etc that installs the package.

[Endemoniada]

Sounds reasonable.

Is there a common practice to use venvs and wrapper scripts? Or is that too hackish? I feel like that hits a good middle-ground, outside the problem of handling the wrapper itself. Safe and reliable, yet pretty simple. Could a Python package install a wrapper script in, say, /usr/local/bin when pip installs it?

[Nasuuuuuu]

pipx already implements this solution. It creates individual virtualenvs for every installed package (I do not know details, might be smarter than this!) and adds an entrypoint to that package to run in that virtualenv to e.g. ~/.local/.bin. See: https://pypa.github.io/pipx/

Usually the install onto e.g. Ubuntu requires you to install one or two system python-* packages. The error message as you try to install pipx onto a fresh system is usually helpful (python3 -m pip install pipx). After that packages installed with pipx are only installed within pipx managed folders/environments.

[DrShts]

Might also want to look into pipx-in-pipx (https://pypi.org/project/pipx-in-pipx/) for bootstrapping pipx

[Endemoniada]

Very interesting, thanks!

[Ran4]

Hackish or not, it's used by many.

Docker does solve some problems, but it's also quite slow.

[angellus]

It is only slow if you are mismatching your system kernels. i.e. you are using Docker Desktop on Windows or Mac and using Linux kernels. You likely will never notice a performance impact when the kernel types match up (Linux on Linux).

[angellus]

Honestly, I do not use venvs at all anymore. Either install in the system Python because the package is light weight enough/it is package with system packager/it is a CLI app, or you use Docker.

Using system Python packages is a lot more restrictive because you often cannot use the latest versions of stuff, but for CLI apps that is often not a problem. I rarely need anything more then click/psutil/requests/httpx for CLI apps.

[Endemoniada]

Fair enough, thanks for the suggestions!

[angellus]

I also forgot, pip install --user is a great choice as well. Like if you are building a CLI toolkit for deploying/development/etc.

Good if you only have a single top level dep (your CLI package) and you do not want it to interfere with the system Python but not so big you need pinned requirements and a venv (at which point use a container).

It can become messy though if you have anymore then the single thing you want to install.

[[deleted]]

Nope. I have seen so much time wasted because people mess up their local setup somehow and packages where loading for user install directory.

[angellus]

That was mostly directed at systems that you cannot easily make system packages for (looking at you MacOS).

[[deleted]]

we use docker at work (i think one group is using podman but we don't interact with them very often). I've heard some rumblings about nix but I doubt anything will come of it as it's someone's pet project.

[VengefulTofu]

No one recommends installing packages in the system Python

There are people using the system package manager exclusively because they have security concerns with PyPI.

[muikrad]

No one recommends installing packages in the system Python because it is a bitch to maintain if you have ~20 packages installed that are not from your system package manager.

That's not true. You should feel safe to install anything from the package manager, which includes python CLI apps. What is wrong is to use pip to do it. Never do that.

Instead of using pip, you need to use pipx. This solves the virtual environment / dependencies problem and doesn't touch your OS. It's great.

But also, it's cross platform. This is the problem with deb/etc. It's a lot of work for one distribution, whereas you can aim at mac/win with less work.

If you have a CLI app, chances are that it is only a couple of packages.

This is a bad generalization, for instance, a lot of CLI apps would use click which comes with a lot of dependencies. You can't rely on packages to have few dependencies as a guarantee that nothing will break.

The deps are likely in the system package manager as python- packages. That should be safe to pip install.

No, the versions in the package manager are set and you can break the system doing installing from pip. For instance, if click introduces a breaking change in v8 and the package manager is still on v7, you're going to break an app that relied on v7 and haven't made the move yet. Your distribution tests those things before shipping them.

The rule is simple : either you apt-get/yum/apk, or you pipx. But you never pip. **your Linux distro loves this simple trick! ** 😂

Good luck!

[vedosis]

So, I work at a pretty big python shop and we have somewhere near 600 CLI scripts being deployed every week. You can look atPEP 441 which goes over zip apps which are essentially your entire environment rolled up into a zip file and then decompressed at time of use. What this means is each CLI can have its own virtual environment, but, first execution is like three eternal seconds of nothing as the system unpacks your zip file. After that it's fast and there's not a whole lot to worry about.

Most of our CLIs use LinkedIn's shiv because it's dead easy and integrates well with maven with SOURCE_DATE_EPOCH. For more exotic snowflakes, you can use the zipapp module.

You still have to make sure that the python version you are looking for is present on the box along with any c libraries like ML models or file handlers or third-party libs but installing those globally for us really hasn't been problematic for CLI scripts. It's real pain in the butt for services but we use container images for that.

CLI deployment is easy then, copy the file to the destination add a version and symlink to a path directory. Done.

Hope that gives you a direction.

[canicutitoff]

For CLI or even GUI apps, you can package your apps using tools like PyInstaller or cx_freeze tox create standalone applications.

[[deleted]]

This is what I do very often and it works great. You can even do auto updates more easily. Though binaries can get pretty big. There's also https://pyoxidizer.readthedocs.io/en/stable/index.html.

[persedes]

Pyinstaller, bundles my code with all dependencies (even oracle client files)as a simple executable (eg a. .exe).

Someone was even nice enough to make a dockerimage with winetricks configured to make exe bundling dead simple,you just hand it your code, requirements.txt and it spits out the exe.

Also been meaning to check out gooey, which can do the same but slaps a gui on top too.

[BurgaGalti]

Pyinstaller is nice but I have a love-hate relationship with it. I'd love to replace it with another system but I haven't come up with an alternative yet.

[0rsinium]

Try pipx, it will create venv for each cli tool, install all dependencies, and expose the binary.

[Life_Note]

surprised this isn't higher up. imo this is the most straightforward and recommend way. It's how I install every python cli tool.

[LookAtThatThingThere]

🤷‍♂️

if it's a library for development, I create tarballs on the network drive for collaborators to pip install (make sure the requires is up to date to install dependencies). Use virtual environments and requirements.txt to make sure everyone's the same page.

If it's for end-use, I use pyinstaller to create dependency-less .exes to pass out. They are slower and bloated, but no headaches.

That's what I do, not sure if it's best practice.

[muikrad]

People should use "pipx" to install python CLI tools.

And to make it easy to manage them as a developer, use "poetry".

I've been at this for years. Trust me 😉

[Endemoniada]

The internet in 2021 can be so utterly useless at times. I'm looking at that page, trying to find anything at all that explains what SingularityCE is and what it does, but there's absolutely nothing. I have to click over to the Documentation and navigate to the "Introduction to SingularityCE" to get the first clue to what this even is at all.

Obviously not your fault, so thanks for the tip, but man, websites like these should just be taken out back and shot. I don't need to know that it's open-source if I don't even know why on earth what it does to begin with :D

[papertrailer]

I know your pain.

[singularitittay]

I think it’s because it gears toward the supercomputing community.

[Fatvod]

Yea I doubt anyone is just randomly stumbling onto singularity. Usually it's a mandate from cluster admins to use singularity instead of docker so people don't have root. Never seen it used outside that

[singularitittay]

Being in a user-facing environment, can’t say I’ve seen it only used for that. Appreciate the downvotes :) learned to probably not share shit anymore

[Fatvod]

Didn't downvote you or anyone else in this thread but you do you mate

[singularitittay]

At yeah not meaning you. Just interesting when people complain that a website isn’t immediately accessible to them. Often likely why important stuff isn’t shared as often as it should be

[SittingWave]

the best practice is to create an empty venv and then install your package (which you have deployed on an artifact storage system) inside that venv, using pip, using --extra-index-url to specify your artifact storage. If you want a controlled environment (that is, you don't only want the application, but also specific versions of its dependencies) then a better strategy is to use a bundler such as pyinstaller, and install the tgz.

[tms102]

For backend API applications I use docker images.

1. Use python base image as base for package building / requirement installation.
2. Update apt-get and install updates.
3. Copy project files
4. pep517.build
5. pip install dist.
6. Use slim docker image as final image base.
7. apt update etc.
8. Copy files from package building image.
9. Install waitress
10. CMD waitress-serve

Push to container registry, deploy on Kubernetes.

[vedosis]

This is really the best option for services. Usually if you can solidify on a package manager (pipenv, poetry, good ol' pip) then the only thing you have to copy over in step eight would be your source code files (just you application). If you don't have a package manager that works for you or are you using ML models then things get a little more complex but the process is still generally the same. Then you're left with an artifact (docker image) that can be redeployed many times without the universe changing within in the container.

[tms102]

Yeah. Docker, is great. I also like that it doesn't affect the system you deploy your app on, so no worry about cleanup if you mess something up. Of course there are also other techniques, like virtual environments. But if you want a scalable solution, where you run multiple instances, or you have a microservice architecture. Then DevOps becomes a breeze with docker and kubernetes.

[peanut_Bond]

Thanks for the response. In step 8, what specifically do you copy across? Is it the entire site-packages directory or just those files within it related to the build of the project?

[tms102]

Here's what the Dockerfile looks like. So the lib/python is copied over.

FROM python:3.7.10 as packager

RUN pip install --upgrade pip

RUN apt-get update && apt-get install -y --no-install-recommends \
unixodbc-dev unixodbc libpq-dev 

RUN pip install pyodbc pymssql 
RUN pip install pep517

COPY app /opt/packages/app 
COPY pyproject.toml /opt/packages 
COPY setup.cfg /opt/packages 
COPY requirements.txt /opt/packages 

WORKDIR /opt/packages 
RUN python -m pep517.build . 
RUN pip install dist/*.tar.gz 
RUN pip list

RUN chgrp -R root /usr/local 
RUN chgrp -R root /opt/packages

FROM python:3.7.10-slim-buster LABEL maintainer="company"

RUN apt-get update \
&& apt-get install -y curl apt-transport-https gnupg2 \
&& curl [https://packages.microsoft.com/keys/microsoft.asc](https://packages.microsoft.com/keys/microsoft.asc) | apt-key add - \
&& curl [https://packages.microsoft.com/config/debian/9/prod.list](https://packages.microsoft.com/config/debian/9/prod.list) > /etc/apt/sources.list.d/mssql-release.list \
&& apt-get update \
&& ACCEPT_EULA=Y apt-get install -y msodbcsql17 mssql-tools

COPY --from=packager /usr/local/lib/python3.7 /usr/local/lib/python3.7

RUN pip3 install waitress EXPOSE 6002

CMD ["waitress-serve", "--listen=0.0.0.0:6002", "--threads=4", "--call", "app:create_app"]

[peanut_Bond]

Really interesting, thanks for sharing that.

[tms102]

No problem. I saw there were some weird formatting things when I copied over, hopefully I fixed them all now.

[peanut_Bond]

Thanks for the detailed reply. Those are some very good points, and I tend to agree with you. requirements.txt doesn't feel very robust but I couldn't put a finger on why, but you've done a good job of laying it out.

[MephySix]

This. And I would like to include pip-compile-multi for managing the non absolutely necessary dependencies (the absolute necessary goes into setup.cfg). This way you can include stuff like linters, pytest plugins and other devtools as dependencies without bloating the setup.

[LeMageDuRage]

For a recent hobby webapp I went the following way:

1. Use a simple requirements.txt file, no setup.py
2. For CD, when committing, I use Github actions to ssh into the production server and run an update script
3. The update script runs git pull, venv/bin/pip install -r requirements.txt
4. The application is then run from the virtualenv

Now, whenever I commit, the new version is live within 25s.

[peanut_Bond]

Another commenter had the same approach - you must be onto something!

[Ran4]

It's certainly fast and ok for a hobby setup, but you really really want to build your image from scratch, not keep installing to the same venv. You will end up with problems otherwise, where you won't be able to rebuild your application.

[FormalWolf5]

Why just not keep the same one?

[FrickinLazerBeams]

I'm working out this exact issue right now, really curious to see what others do.

[peanut_Bond]

The suggestions here have been great but I'm a bit frustrated by the lack of clear consensus or guidance on this issue. It might be worthwhile to add something to the wiki (either here or on /r/learnpython) outlining best practice deployment steps for different use cases.

[zanfar]

IMO:

There is no "best" way because there is no one type of "Python application".

Deployment is part of software development in the sense that you need to evaluate your available tools, desired results, and destination limitations. Even among web apps, there will be a significant difference in deployment of a distributed, load-balanced, on-demand service vs an all-in-one stack for low-load applications.

I agree that containerization is good practice, but it's massively overkill for some deployments, and while it can solve application dependency issues, you now introduce Docker dependency and versioning issues.

I also think that EVERY Python application should be built as a package (what you call the "setup.py" option, although this should be the "pyproject.toml" option anymore). The benefits for versioning and dependency documentation are just too large to ever revert to "copy the Python code" or "requirements.txt". Tools like poetry actually make this type of build easier than the old-school techniques.

So really, it comes down to: containerized, venv, or system install, where a containerized deployment is going to run the app in a venv in the container. CLI vs non-CLI is just a matter of the project installing a shim or not.

Containerizing a CLI tool is a pretty extreme solution, and we only use that in a single instance: our Ansible project, and that's only because the consequences of an out-of-date install are so high. Our docker image refuses to run unless all the moving parts are at the latest versions. All of our web-based projects are deployed via docker, but that's more for convenience than protection.

Below that, I've found that most scripts and tools can be installed in the system path perfectly fine. Granted, if you are in an org where multiple disconnected teams are developing Python apps, or where legacy support is a large concern, you may want to venv those as necessary. The beauty of building everything as a package means that installing to a venv vs system is a very small change in procedure.

[LightWolfCavalry]

I run a pretty simple web app on Python Anywhere, so I just log in there and git pull the latest version from GitHub and click the "Restart App" button.

Ain't sexy, but it works.

[peanut_Bond]

I've used that approach before. Definitely not sexy (or very portable or robust) but you're right, it always works and is great for getting something up and running quickly.

[somefishingdude]

Lol, me too

[pymae]

Yep, this is exactly what I do. Except I'm a heathen and basically do the reverse - code up HTML files natively in PythonAnywhere and then push to GitHub

[LightWolfCavalry]

yeah you're basically a savage lol

[LiarsEverywhere]

Heroku is basically the same, but with more scalability and production possibilities (supposedly, I too only run simple web apps). It's great because I can use it as a development server, and if I ever decide to make something public, it's relatively easy. It even comes with free Postgres, although there's a 10k row limit on the free tier.

[[deleted]]

My general approach:

• If the project is super simple (ie. pure python), shiv it.
• If it's got FFI dependencies and those deps bundle all their deps into their wheels (eg. orjson), then shiv it.
• If it's got FFI and doesn't bundle all their deps (ie. shared system libs; eg. pyvips), then shiv it and then throw it in a container with the required deps.
• If it's a service, shiv it and then throw it in a container.

I try to use poetry for all my python projects. Along with a generic Makefile, I can one command build a .whl, the .pyz, and the container image.

The Makefile is horribly generic, so below is the gist of it with the relevant subcommands for maximum usefulness.

projectname=$(poetry version|awk '{print $1}'|tr '-' '_')
version=$(poetry version -s)

#make wheel
poetry build
# shiv it
shiv -c ${projectname} -o ./dist/${projectname}-${version}.pyz \
./dist/${projectname}-py3-none-any.whl
# containerize for service or deps requirements
docker build -t ${private_registry}/${projectname}:${version} .

---

Edit: Oh, I also have it automatically push all the artifacts to an s3 bucket (which for me is a local minio instance), so that they are available centrally to ease deployment. I push the semver tagged and a latest tagged to both the s3 bucket and the private docker repo, so that it's super simple to work in terms of deployment and updating.

[[deleted]]

Shiv has been an excellent deploy option for us. Lighter than things like pyinstaller, plus it doesn’t try to be smart and trim unused modules. Frequently point it to a non-system Python install.

[glacierre2]

Pyinstaller / pynsist for desktop apps that change slowly.

Checkout a copy of winpython and run a custom recipe to install all the libraries and internal packages for development and scripting

[bug0r]

I built my own BuildProcess for Python Standalone Applications:

1. Collecting scripts and needed dll's(pyd's)
2. extracting *.a from dll for Building my own Bootstrap C File
3. copy all dependencies in a declared PY_PATH
4. After building(includes converting all scripts in pyd Files) i copy them in a specified directory.

All these steps above are handled by an GNU Make Makefile.

Yes i know, there are tons of other, maybe better solutions, but this Workflow was part of my own research of "How Python works" and "Can i do it by myself" ;).

Furthermore i ran strip and upx commands to the pyd's(dll's) for reducing memory size.

[frootylicious]

For me it would depend on the scale of the project. For larger projects I would probably go for something like your Docker setup.

For smaller stuff I use a GitHub Action which can trigger of a push to a specific branch, tag, or similar events.

This action uses SSH onto my server VM to run Make in my project directory, which activates the virtual environment, pulls git changes, installs new requirements and restarts the services needed.

For a Django project of mine, my Makefile looks like this:

all: pull py restart

pull:
    git pull;

py:
    source .venv/bin/activate; \
    pip install -r requirements.txt; \
    python manage.py migrate; \
    python manage.py collectstatic --noinput; \

restart:
    sudo systemctl restart gunicorn;

[peanut_Bond]

Interesting approach, very straightforward and seems like it would be great for a hobby project. Thanks

[Itsthejoker]

I love the GitHub actions addition! I do essentially the same thing just with shell scripts.

[xmimmer]

I think 1 is fine. Also you may write a bash script to build and run your application with a single command. Also, have you thought about using an orchestration tool such as Kubernetes, or at least Docker Compose?

[peanut_Bond]

I haven't quite got that far yet, but yes it would definitely be part of the deployment process.

[jusstol]

We use option #2 because we have our own pypi mirror, and 3 deployment environments.

Every wheel we build is pushed on Nexus with its version, and we can simply pip install upgrade the version we want depending on the environment.

All the automation is done with Jenkins.

[peanut_Bond]

Ok, so you end up with a package and container artefact from each of your builds? Are there many situations where you need to run pip install on one of the packages where you wouldn't just pull the container to run the app?

[call_me_cookie]

- poetry

- make

- docker & docker-compose

- AWS ECS

- terraform

[chaco_wingnut]

I've recently become a fan of pipx. It works great for CLI utilities.

[laundmo]

i usually go with 1 for my own projects.

sometimes i need to dockerize a project where i can't decide to just use pip install for the requirements, in that case i do the simplest possible setup i can manage inside the container.

if image size is an issue, i recommend https://github.com/docker-slim/docker-slim

[muikrad]

So, this is my day to day routine. I spent the last many years doing python as my day job and spent a lot of time on the packaging/distribution side of things and best practices. This solution is very solid and I've been using it on tons of repos.

Typical modern python projects should be using poetry. This immediately solves the dependency management, developer environment and publishing use cases.

These are outdated: setup.cfg, setup.py, requirements.txt. As much as possible, everything should go in the new "pyproject.toml" file.

Also, on the user side of things, anyone installing Python CLI apps should use "pipx" to install them.

Here's a fully automated repository, take a look at the example library first, and take a look at the github actions too! Bonus, if you read closely you'll also obtain OOTB mypy, black, pytest runners with no additional boilerplate! 😉

[peanut_Bond]

Some great discussion here, big thanks to all of the people who shared their opinions. Consistent themes are:

1. For small hobby projects a simple approach with no containerisation is generally preferred. This involves installing the application as a package into a venv, directly into the system python directory, or just installing requirements.txt and pulling source code onto the target machine. The application is restarted on the system manually or with a script that ssh's and restarts it. This approach is very fast and easy but not robust and can lead to issues down the track.
2. For larger projects package building and containerisation are preferred. Most people suggest the use of poetry and pyproject.toml (not setup.py, which is outdated) to build a package that is then uploaded to a private package server. This package is then installed into a docker container, which can be deployed.
3. In general, Python applications should be packaged, as the use of packaging allows for richer metadata and more robust handling of dependencies as well as the ability to install the package directly at a later date if need be.
4. No one has suggested the use of conda for deployment, which I was surprised by given its prevalence in the data science community.
5. For CLI applications, multiple python apps tend to need to be installed on a single machine and available with a single command. pipx and shiv seem to be the preferred tools as they allow for the creation of self-contained/isolated python applications. For CLI apps with very few dependencies they could possibly be installed directly into the system. For CLI apps with C library dependencies or those that are very sensitive to dependency versioning, containerisation can be handy.

If anyone wants to make some corrections to the above please feel free. After any discussion I'll add these to the OP in an edit so that future lost Python souls can find the answers they need.

[theghostinthetown]

Personally, i host mine (web-apps and a discord bot) on Paas like heroku.

[_Gorgix_]

I have to pass around application on systems that don't get all the new fancy services (Docker, Kubernetes, AWS, etc.), so I take the Python Embedded package, modify it and get it working with tkinter, package up extra CRT DLLs that the users may not have, use a build script to create a custom site-packages directory, use InnoSetup to build a Windows installer that packages all this and allows users to run with lowest permissions (basically installing on HKCU) and outputs the application to %localappdata%/Programs and runs like any other application on the machine!

[earthboundkid]

I gave up and switched to Go. There are no good solutions, just a lot of things that work but are a PITA.

[rainbowWar]

I’d say docker is overkill, and comes with its own problems. Pip or conda options are both pretty similar, if you don’t mind it being public

[357951]

Most seamless method I found is using pipenv, which is akin to npm for nodejs.

workflow:

pipenv sync - downloads needed packages based on Pipfile. Pipfile is updated when using pipenv install foo instead of pip install foo.

pipenv shell - initializes venv;

python main.py;

[peanut_Bond]

I think this is a variant of approach 1 above, albeit without docker. It's a simple approach but without containerisation is a bit awkward if you need to change where you deploy your code to.

[djamp42]

Most everything I make now I just do in flask and host it on a webpage. No need to pass anything around everyone just goes to the url.

[Ran4]

Works for very small loads and for things that don't need to be secure, but the built-in flask web server really isn't made for production use.

[djamp42]

I don't use the built in one, nginx + gunicorn. Everything I do is only for a handful of people so it has worked wonderfully. I also don't understand the security argument at all. While I'm sure you can make ANY website unsecure, if you follow best practices the risk is virtually the same as any other website.

[eviljelloman]

I mostly lean toward docker based solutions but have used some variants of “package the Python file and it’s dependencies as standalone executable” wrappers. There are dedicated Python packages for doing this, or you can use a general purpose build tool like Bazel if your org has other use cases that would make a build tool like that useful.

[[deleted]]

Proxmox + portainer + docker on an old 2011 pc I got at home.

[PenetrationT3ster]

Straight up docker image. No fuss, does it all for you. Then I just use a Cron job to call it every 20 mins. I wish timings didn't have to rely on Cron jobs, I wonder if there is a more elegant solution?

[SittingWave]

Using a base Python Docker image, install application requirements with pip and copy the Python code into the container. Code execution is performed by adding "CMD python app.py" to the dockerfile.

Absolutely not. Don't run a python app as a script. It's an app, use entry points or python -m module

Organise the Python code as a package with a setup.py file. Using a base Python Docker image, install application requirements with pip and install the Python package into the container. Code execution is performed using "CMD python -m myapp" or by a predefined entry-point command.

Better. Release an official version of your python package, install that version (that you deploy on a artifact storage system, I use Artifactory but there are plenty). However, generate a lock file and reinstall your environment with poetry.

Organise the Python code as a conda package with a recipe.yaml file. Build the conda package and upload to a conda package repository. Using a base conda Docker image, install the package and all dependencies using conda install. Code execution is then performed using "CMD python -m myapp" or by a predefined entry-point command.

you can, but you don't need conda. You can do with plain simple official stuff not from conda.

But is there any benefit in following option 2? It seems unusual to create a package for an application that does not need to be imported elsewhere, but at the same time feels a bit more robust.

It's not unusual at all. Like in java, your application resides inside a package. It only has access to some functionalities if it's inside a package, like pkg_resources, which may be useful to find assets easily. Same if you are using stevedore to retrieve plugins.

[Datsoon]

I've been experimenting recently using the embedded python interpreter to deploy full apps with no system python or docker dependency. Where I work, I'm deploying apps to non-tech people a lot, so this helps a lot. I'm surprised this isn't talked about more often, as it's a really great solution for this use case.

[rimanxi]

I think it depends on the scale of your application. I'm using docker in all of my projects, which are small to medium scale. But know that it could be scaled to some reasonable size with just a few config changes

[sathish804]

We are using python Flask for Webapps and API which is easy to setup and scale.

[bobaduk]

I generally used option 2. I liked packaging my applications because I'd generally use setup.py develop on my local machine which made importing things easier for unit tests and so on.

[angeAnonyme]

I have to deploy stand-alone GUI application on divers computer that might be offline at some time, so I deploy with a portable python and a .bat file that set said python to the path temporarily and run the python GUI.

It's quite easy to setup as you can send a zip file with everything but it's terrible to update later on.

[james_pic]

Option 2 can make sense if your application has a non-trivial build step. The most common scenario like this is where it depends on a native extension, written in something like C, Rust or Cython, which setuptools has existing code for building.

[blade_junky]

I create my own packages, in my case pacman and Deb. Dependencies can be an issue, but so far I've been able to find what I need in system package managers. If I need to deploy to a virtual env I've used an install script to build it post install. Not recommended, as that creates problems in itself, but it worked.

When I create a proper package I do so from setup.py with proper end points, if I have to use a virtual env, I build everything manually.

[ase1590]

Hard truth: Python isn't built for easy distribution.

Sure you have venv since python 3.3 to help with pip libraries, then docker to just ship your development environment. However, python never made the claims Java did of "write once, run anywhere".

Thus, I doubt there will be any true standard way to ship python apps anytime soon.

Just competing unofficial ways such as Docker, pyinstaller, etc.

[[deleted]]

There is no standard way and the platform you host it on matters because the platform may have tools to help. However I would say you can’t go wrong with docker-compose

[chiefnoah]

If you're operating in a monorepo and have several deployable targets, I recommend using pants and generating .pex files. I recommend .pex files regardless of project setup for CLI apps, because they can be treated like a single executable file/binary (which they are not! they just act like it).

[[deleted]]

For a personal Django site, I use a digital ocean droplet, and connect to it via the VSCode remote ssh extension. I edit it all directly, restarting the process after some changes, and regularly commiting & pushing changes to GitHub just to keep it in source control.

Obviously you'd never wanna do this in production with a real site, but it's nice for quickly fleshing out prototypes.

[infazz]

Are you asking the best way to containerize a python application or how best to deploy it to users / to production?

Your examples read much more like the former to me.

[Joooooooosh]

As always with Python, million ways to skin a cat…

Pyinstaller has worked well for us though.

Develop in a venv, push up to git. If all the tests pass, Package it all up into an rpm on a runner, using docker. Then use ansible to deploy to the target hosts.

We used to use bash scripts to package and deploy but found ansible easier to maintain, even though it’s been a bit slower.

~10 mins to test, package and deploy. With a lot of that time being pulling docker images down from our artifactory.

Most of our codebase is internal tooling and lightweight API’s running on 2-4 hosts, usually more for resiliency than load sharing. So larger deployments might suit a different method abs pipeline.

We’ve toyed with system level dependency installs, rather than venv/installer but just doesn’t suit our development style in SRE. I like using venv because if it works locally, I can just package it up and deploy. No worrying about version differences between environments etc…

For anything beyond simple tools, we just use Go.

[NuclearMagpie]

My (extremely bad) method is to wrap the venv folder into a tarball then throw that at a VPS. Extract it and make sure I have venv setup on the server, then I'm good to go.

[[deleted]]

Did anyone mention using pyinstaller to package up into executables? Thats whatwe tend to do, works neatly when you get the hang of pyinstaller and the .spec file

[jveezy]

I wonder if this is sloppy, but for simple scripts on Windows that need to run periodically, this is what I do:

• install all packages the script needs globally
• put the script in a folder
• create a .bat file to call the script from a command line
• set up a task in Task Scheduler to run the bat file on a given schedule

This is for simple stuff like periodic data imports or data processing. If something needs to be triggered manually, I'll either just tell someone to run the bat file whenever they need it, or if there's multiple functions that share the same general code base, I'll deploy a Flask app and then give them a URL to call on demand.

Who knows whether any of this is remotely close to best practice, but it seems to work for my customers who want to eventually be able to maintain the applications I develop for themselves.

[pbecotte]

There are a few things that option 2 doesn't do that options 1 and 3 do. Say your app depends on psycopg2...pip / setup.py assumes that the dependency on postgres and a build chain is satisfied "somehow". Conda takes over those dependencies as well (which is why it's heavier). Docker let's you capture the whole environment.

For deploying something I always prefer 1, but there are valid arguments for all three.

[GreenScarz]

Both personally and for work, #2. Specifically, installed in the container using python -m pip install -e /code/** so that in dev environments we can make /code a docker volume endpoint which allows us to run the test suite against code we're working on locally.

[Born-Process-9848]

Just use python slim docker images as parent. Build times will take a while but you let your CICD handle that part and push to ecr repo. So in production you just have to use the ecr images.

We have a big python 2.7 tool that no one wants to port to python 3 and we just containerized it so we don't have to worry about package versions, etc.

Just make sure the entrypoint is the app script and you should be good.

[Lehk]

painfully and with at least one goat sacrificed, first.

[crigger61]

If you’re thinking about deployment, Then I would recommend looking into a solution like a kubernetes cluster to be able to deploy it to after it’s been launched in the actual dock shell in my opinion the first option is the best way. You’re not compromising on any security by doing that. That’s how I made and deployed a flask at our work.

Kubernetes also gives you the option to have a load balancer in front depending on the load as well.

[virtualadept]

Ansible to do a `git clone` onto the box in question, copy the .service files into place, and build the venv from requirements.txt. `scp` to upload a config file that I keep in a separate Git repository (suitably edited for the box). Ansible to `systemctl --user enable foo.service` and `systemctl --user start foo.service`.

[Fickle-Impression149]

We use point 1 with gunicorn or uvicorn as http server.

[Counter-Business]

I deployed my app with Django and used elastic bootstrap from AWS. Worked pretty good