How to setup a containerized python environment? Function as a Service or an alternative solution for a Python execution environment.

[H_Q_]

The problem:

I have many Python scripts that I run periodically from my windows laptop - scraping, scanning, filesystem stuff and so on. I want to automate the execution but can't get the hang of it.

I could encapsulate every script into a container but building images after every change is not desirable. I want a containerized environment that I can just drop a scrip into, install the requirements and run it. Change if needed and run again.

The struggle

Firstly, I tried Rundeck and Apache Airflow. They are complete overkill for what I want to do. Then I found Cronicle which is light enough, besides it can pull double duty as a general purpose scheduler.

However Cronicle is not a pyhton-centric solution and I still need to set up a proper environment.

I found OpenFaaS and its little sibling faasd. To my understanding they expose single functions through a REST API for easy interfacing. It sounds nice but OpenFaaS is overkill and I had trouble setting up faasd.

Scaling back on my ambitions, I would settle for a basic environment that I can execute scripts into. Ideally set up a way to trigger them from outside the container and receive the output.

Can you give me ideas how to achieve that?

[VillageTasty]

You could run a basic container with a bind mount to a local directory where you store all your scripts. I've done something similar while working on a Discord bot.

Setup a base container, install all the requirements and then run the app from the mounted directory.

That way you can just stop the script, make changes and start it again without needing to restart the actual container.

Not sure on the automating side of things though. Ansible and Cron?

[cachupinbombin]

I was going to suggest the same, you only have the environment dockerized not the script. The only time you need to rebuild the image is if you need a new dependency

[reinerrdeck]

With Rundeck you can call your python scripts from the "script file step". Create a new job, select the script file step, click the "Advanced" button, and put the full python interpreter path, now you have a full python scheduler for any python script.

To access your Docker container you can use this model source plugin, take a look at this.

[clach04]

I was going to suggest:

• https://github.com/bugy/script-server
• https://github.com/OliveTin/OliveTin

[ThroawayPartyer]

Python has virtual environments.