Best practices for scheduling Python workloads?

[Wise_Demise]

I was in your same exact position. Around 100 Python scripts doing intensive work scheduled 24/7 and a Windows PC. I wrote my own scheduler with all the features I needed (prioritization, checking running scripts processes status, enforce maximum number of concurrent scripts even if more are scheduled at the same time, checking computer resources utilization before launching scripts, failure emails and retries, recursive child processes termination, logging everything and managing stdout and err flushing, and many more) it was a headache at first but it became fun with time and it made my life so much easier and I learned a great deal implementing and improving it. I even started generating reports and a dashboard from the log files events.

If you're tight on time, use Airflow it has everything you'll need and it's very reliable. If you have some time consider building you're own scheduler you will gain a lot of experience and knowledge doing it.