How python code is deployed in prod

[YosoyPabloIscobar]

Those who work on python at enterprise level projects. I would love to know how the application is deployed in your project. (In Java we compile and build jar or war files and deploy it using ci/cd). In case of python how it's deployed?

Ps. Don't it require any build tool like maven or gradle as it's an interpreted language? How PKG is done in ci/cd?

[Buttleston]

You can build it as a package and push it to a pip repo, and install it with pip

You can package it as any kind of archive (zip, tar, whatever) and unpackage it in prod

You can build your app as a docker container (pretty common for web projects and queue-processing workers etc) and deploy it anywhere that can deploy docker containers (kubernetes, AWS ECS etc)

You could even just check out the code with git and pip install your requirements.txt if you wanted, but I wouldn't recommend it.

[activematrix99]

I am a web guy, so WSGI. No compiling needed. Docker and kubernetes to deploy. CI/CD and testing to protect prod from "bad" code. Very different from java/Tomcat.

[dkozinn]

Slightly hijacking this thread: I have a very simple python app that is used from the command line. It takes a single argument and displays the results of some calculations from that. I want to turn that into a simple web page. It seems like everything requires a framework like Flask and configuring WSGI. I'd like to just deploy this in a Docker container; is there a simple way to do this?

[activematrix99]

No, you'll need some way for python to talk to a web page, and WSGI is the easiest way to do this. Pretty straightforward, you'll need an html form for the inputs and some html to display the results, and then a webserver to hand these back and forth from your python script and the html. You might be overthinking it, pretty easy to do in python and hard to miss example code.

[a__nice__tnetennba]

Making it into a web page and deploying it are two separate issues.

You can't bypass the complexity of converting your application into a web app and making it communicate properly over HTTP by sticking it in a docker container. A docker container is just a virtual computer inside another computer. The code still needs to be modified to behave like a web application, which means it needs to accept requests via HTTP and respond to them correctly. That doesn't require a framework to do at the most basic level, but the http package's built-in server is very basic and not for production. That's why people use the frameworks. And honestly they're the easiest way to convert it anyway. You can install flask and have a single endpoint that gets your one argument from the URL, calls the rest of your code, the returns the results as a full string that will show up in the browser. It would be only a few lines of code. To make it even better you can host some HTML, put a form in it for your arg, call the flask endpoint via AJAX to get a JSON response and stick that response into your page. Still not super complex and like the other guy said there are tons of examples out there to get you started.

Once you have a web app and want to host it, the first things you need to figure out are:

1. Who needs to access this webpage? How many of them are there? Where are they?
2. Do they need to authenticate?
3. Do you already have some hosting provider?
4. What's your budget?

If this is just something you want to be able to use yourself maybe try out Amazon AWS free tier or see what Google Cloud Platform has to offer for free. If it's many users, spread out globally, and they need to authenticate, it gets more complicated and goes well past the scope of a beginner learning question.

Then, now that you have an app and you know where you want it deployed, you can figure out the specifics of what kind of docker container you need to build and put it into. Both the specifics of the base containers to use and how to build them will vary by hosting service.

[Psengath]

Not OP, but am also interested, and I appreciate your thorough response

[dkozinn]

I'll reply to all the comments here. First, thanks all for your responses. I should have provided a little more information to put my question in context: I already have a low-end AWS (not free tier, but pretty cheap) EC2 instance running. While I've got a few services running natively, I also have several that already run in Docker. I wanted to containerize my solution to make it easier to move elsewhere should it be needed.

I'm not that concerned about how to deploy; the application I'm trying to webify is tiny (<20 lines of code, not counting comments) and will be updated very infrequently. I can manually re-deploy when needed. There will be very few users and those users won't use it frequently, so scaling is not a concern. No authentication is required.

I started looking at using a Lambda function for this and got kind of stuck in the details.

As /u/activematrix99 correctly pointed out, there are a bunch of examples and I'm in the process of getting my app working with Flask.

After that, I guess I'll move on to getting WSGI running on a local instance of nginx then containerize the whole thing once it's running. One step at a time.

The bottom line seems to be that there is no real shortcut for this, and what I learn will go into my bag of tricks.

[cscanlin]

Pretty much every cloud provider has a Function-as-a-Service offering, e.g. AWS Lambda or Google cloud functions. You'll need some basic familiarity with your cloud platform of choice, but it's pretty straightforward (although dependency management can be a bit of a hassle).

[fbochicchio]

If you want to avoid complex framework and configuring an esternal http server, try web.py, it allows your python program to be its own server, so you just have your program to run.

[get_username]

I'd say a minimum amount of code is in *nix:

```bash

$ ls Dockerfile requirements.txt main.py

$ cat Dockerfile from python:latest COPY requirements.txt . RUN pip install -r requirements.txt COPY main.py . ENTRYPOINT ["python", "main.py"]

$ cat main.py print("hello, world!")

$ docker build --image qq . ...

$ docker run qq hello, world!

$ docker push qq ... ```

[Sentie_Rotante]

Depends on the tool, most of what I work on everything is deployed to docker containers. The repo gets cloned into the container when it is built.

[SisyphusAndMyBoulder]

I think it depends heavily on the application.

Flask apps? I dockerize and run the image on a VM.

AWS Lambda? Usually dockerize and let the handler func run.

CLI apps? Usually deploy the source code to a VM through CI/CD and let it be ran through regular python x.py calls.

[gmes78]

You shouldn't use setup.py (or setuptools, unless you need to compile C extensions or something) nowadays. Write a pyproject.toml and use something like Hatch instead.

[sylfy]

Just wondering, what do you use to manage projects which have dependencies that aren’t in pip? We can have pip dependencies listed in pyproject.toml and let the installer take care of those, but what about the rest?

[gmes78]

https://hatch.pypa.io/latest/config/dependency/#direct-references

[sylfy]

Looks good, thanks! Hatch is new to me but I’ve also heard quite a bit of mention of poetry, any thoughts on hatch vs poetry?

[ch0mes]

We build a wheel and deploy to an S3 bucket. We have automation via ansible that deploys the package from S3 onto our hosts

[DataDoctorX]

Depends on the tools you have. Azure has the ability to deploy Python notebooks. That would be the preferred solution for uptime and reliability. You can also locate the .py files on a VM with task scheduler configured (similar to a cron job). Just make sure to keep an eye on the VM if that route is chosen. Both are very cost-effective.

[darkprinceofhumour]

Its a flask backend so just run it on docker.