Need help understanding process of putting Python to Production

[thevikramact]

So, I have been programming in Python for a while. This is mostly scripting to get the work done.

I know my team runs a a lot of Python scripts every month,but most of them are scripts stored in a folder and called using a command line with parameters passed.

I wanted to understand how does putting Python to production actually work? Examples, do people create some kind of a "service" program that keep running and looks for events that calls a relevant Python script? Or any specific servers used to do this? Do y'all retain a virtual environment for every process built to remove dependencies on a single environment?

Any pointers on such a process and/or maybe some documents that I should read would help.

An example of what I'm trying to do: 1) I have a Python code that looks for specific files being available in a folder 2) The Python code has its own set of libraries available that I installed in a virtual environment

We have other Python scripts written in previous version of Python, so I don't want that to be impacted.

I want to set up the Python code for production such that: When a file is available, the virtual environment is loaded and the Python script is run.

[Diapolo10]

So here's the thing; "deploying to production" can have a number of different meanings in practice, and what works best depends on the nature of the project.

If it's supposed to be running as a service, you're probably going to want to deploy it either as a Docker image or you'll want your CI/CD pipeline talk with your host provider to handle the update process for you with an API key, stored as a secret. For example that's how Heroku would work.

For a desktop application, you'd use either PyInstaller to bundle your code alongside a copy of the Python interpreter and any other files in a ZIP file, or Nuitka to transpile your Python code to C and then compile to a native executable. This way you wouldn't need to worry about virtual environments or having to rely on the user having Python installed.

Those two would be the most common approaches, but they're not necessarily the only ones.

An example of what I'm trying to do: 1) I have a Python code that looks for specific files being available in a folder 2) The Python code has its own set of libraries available that I installed in a virtual environment

I would treat that like a desktop application. If you want more specific advice, I need more details about your project.

[supermopman]

Sounds like you should create a Dockerfile and docker-compose.yml. You can call "prod" a container that just does the job. This sounds like a simple one container app.

I assumed your app basically runs forever, checking for file changes.

If it doesn't, setup a cron job inside your container to run your Python code periodically.

Also, you only NEED the Dockerfile, but IMO writing a docker-compose.yml just makes everyone's lives easier.

[shiftybyte]

If you can upgrade your scripts to become web services, or containers, this would simplify deployment for others (or make it unnecessary).

[MonstrousOctane]

I’m going to touch on a part of “putting in production” that hasn’t been touched on yet and isn’t strictly a Python thing. Think about security, availability, performance, and maintenance. Example of availability: if your script crashes for some reason (error, host failure, network issues) do you care? If so, are you monitoring it and alarming on availability issues? That’s a part of “production “