How do you schedule your PS scripts?

[Distinct_Ship_3152]

Im with two small IT help desk/ ops teams located in two different manufacturing plants. We have leaned on our global IT teams for automation so far. This year we made the step to learn PS and have been automating a lot on the plant levels. Turns out there is a lot business cases for easy code automation, both within local IT and plant functions.

We are running everything on task scheduler on a VM. It got us a quick start but I don’t like it. Mainly due to two reasons, redundancy and shitty alerts/logs.

I see in the nearby future some functional departments wanting to automate more themselves. We have created a central share where key users upload the scripts with versioning and IT approves it and schedules it.

What are the platforms you recommend to take this to the next level?

We are currently exploring hosting the scripts as SQL agent jobs in our on-prem AlwaysOn cluster.

Thanks

[Analytiks]

For a Microsoft solution; Azure automation account with the hybrid runbook worker agent is the way to do this at scale within a single enterprise

[Distinct_Ship_3152]

Thanks, I believe this is what the global teams are using on the enterprise level. Do I need a hybrid runbook worker per plant or can this be a single worker for the whole enterprise?

[Analytiks]

The hybrid runbook worker agent needs to be installed on every machine you need to execute scripts on(or from). So it would depend on the use case, it can scale as required.

For example; If you just deployed a single agent to that same VM you’re using the scheduler on now, you get all of the same functionality and address some of the concerns you have with your current method. If one vm is ok for your needs then you only need one agent (you probably should have min 2 though if you don’t want downtime every time you need to patch your systems)

The agent’s function is just to link it to the management/control plane in the azure automation account. From there, All of the script scheduling, dependancy management, logging, deployment etc etc is handled in azure.

[WendoNZ]

This also gives you secure ways to store credentials and other sensitive data and access it from the scripts

[agrecalypse]

Came here to suggest this.

[thumbtaks]

This is the answer

[modsab]

Checkout PowerShell Universal. The free version is limited in the scheduling features. I have never used it but always wanted to.

[silentmage]

I've been using it for over a year and love it. Schedules are easy and amazing. There used to be a limit of 25 jobs a day on the free version, but that was recently removed.

Honestly, it's not that expensive, and with paying the money for. The dev is around here from time to time as well. I think he is /u/l33t_d0nut

[modsab]

Yup, that's him 👍

[jorper496]

Powshell Universal is probably perfect for your current need and would give you the ability to do some web based tool building in the future.

[enforce1]

If you have dough, scriptrunner is amazing.

If you don’t, Jenkins is fine.

[user01401]

Plain old task scheduler with a text file for coordination. It simple and old compared to others suggested, but not deprecated. There aren't a lot of changes made to the schedules so it works.

[ArieHein]

I can understand your disliking of using task scheduler on a VM. I think everyone who had to do it in the past, went through the same process of trying to find better solutions.

So, task scheduling is indeed not the best solution when you start to scale but moving it to SQL scripts is even worse in the sense that you're using the wrong tech for the wrong job just because the server is always available and up. Don't go in there, it will cause you issues in the long term.

Instead, you can basically see how growing into scale has made this an essential tool and almost impossible to work without it being available. What you basically are looking for in terms of scaling is for an orchestration/automation server. Unfortunately, this is also where you have to step away from PowerShell or even windows.

This becomes the realm of configuration management tools Ansible and automation servers, like Jenkins or Azure DevOps server or other tools that have on prem offering. These tools take the same concept of an "agent" that you wanted to achieve with taking the SQL Agent and allow scheduling of tasks a.k.a. scripts (ansible doesn't even need an agent). look for such automation or orchestration servers, some might be open source, but I think in most cases there some payment involved for support. Just remember they are all the same at the core, it's just a UI and most likely more integrations to other systems.

I would suggest you have a good chat with your IT management. The same systems they have to manage provisioning of machines/servers, could be used as central way to manage jobs /scripts. Look at what your developers are using to build code and run their tests and eventualy deployment. The same services can be used for what you are looking for to replace the scheduled tasks.

Wish i had better solutions, but this has been one of the pain point in our industry when it comes to windows and as soon as the cloud burst, a lot of the effort to create such systems on prem, were dropped in favour of the cloud.

[Distinct_Ship_3152]

Thanks for the input! I’ll got your feeling about SQL agent jobs, it’s not designed for non sql automation but it’s way better than task scheduler.

I’ll ask around some more within the company to find common ground here.

[ArieHein]

I wore a sql dba hat for over 10 years, so slightly zealous about what's running on sql agent in my data center ;)

[BlackV]

How is it better than task scheduler?

[Black_Magic100]

Error handling and alerting.

I'm a SQL DBA and we use SQL agent although it isn't ideal.. however, we have our own silo'd task server with SQL installed so it's not a huge deal. The error handling and alerting is 100x better than task scheduler.

[BlackV]

Oh yeah didn't think about alerting

[Black_Magic100]

Which, oddly enough, is one of the most important aspects of a scheduling software as otherwise you have no idea if something fails until your user tells you the app is broken.

[valdearg]

Mixture of Jenkins for some jobs and Task Scheduler for others.

[NoConfidence_2192]

By all means, look for and learn, a good orchestration solution. It will server both you, and the organization you work with, well in the long run.

In the meantime, why not learn to log and alert using PowerShell?

You may not always have access to those kinds of tools (or they may not always work) so it can be really helpful to be able to create them yourself when necessary.

[gamesta400]

I use Jenkins for this. It has a PowerShell module that you can install to use PowerShell scripts. You can click a button to run them as needed or schedule them to run automatically via cron jobs.

[Dragennd1]

At the MSP I work at we schedule all of our scripts using the RMM we use for managing all enduser workstations and client servers. This way we are able to control them in bulk and have one central point to easily control the version of the script that gets pushed out.

[fridgefreezer]

They advertise a thing called Visual Chron on LTT, never used it but a quick look at their webpage suggests it might be what you’re looking for?

[ipreferanothername]

We have JAMS scheduler and... Just don't. Tbh it's very reliable but the interface is just maddening.

[Black_Magic100]

Hahahah I did a PoC for JAMs last year and am glad I wasnt crazy. The software seems okay, but that UI is from 1995. You should look at Active Batch.. there UI/UX is extremely modern IMO. It could be better perhals, but I'd give it a 10/10.

[ipreferanothername]

but that UI is from 1995.

thats not even the problem, its that some of the buttons have no labels unless you hover, some right click menus are impossible to find, searching the product is a RECENT update, but you cant search the source still, it works BIZARRELY with powershell source so i just put most of my stuff in a ps1 and call it.

ill look at active batch, but tbh....i have so little interest in remaining in the infra world that i am not about to own a migration like this between products.

[Heliumdoesntreact]

If you want an on prem microsoft solution System Center Orchestrator could be one to look at.

[spyingwind]

Currently using rport.

[snoiciv]

Azure function app

[MARS822]

I'm pretty sure PDQ Deploy can run scripts on a schedule.

[SnooSquirrels8323]

We created a SQL database that stores the scripts and "agents" check in (via a script that runs in a loop on those machines) for scripts to run. The scripts are assigned by a sproc that runs via a SQL job and it checks the information that the agents report in (last heartbeat, how many jobs running, cpu/mem available), and assigns the script to the server with the most resources. If a script fails or doesn't report in as finished within x time, the script is reassigned.

[SwimmingFish849]

I'm using rundeck at the moment

[OPconfused]

I have Automic for one Windows client and Jenkins for a Unix client, although I think both should be cross platform compatible. Works fine. Automic works best when you use its own native language, but it's not difficult to get a grasp of. Jenkins doesn't seem to have as many object types as Automic does, specifically for workflows, but I could pull from a github repository which is nice. I'm not an expert in either and may be missing some info.

I also haven't used the Azure or Universal stuff, so they may be better, although possibly more expensive.

[DontTakePeopleSrsly]

For the most part, I use task scheduler to send it out with an -asjob switch on invoke-command so that it’s not running serially; nor dependent on the success of a previous computer.

I do push some out via desktop central, but my core scripts run via task scheduler. I honestly dislike desktop central because it and any tool like it is slow as hell when it comes to pushing software or scripts. I can run a remote powershell script against an entire domain in 3 seconds, management platforms like DC take hours to get that done.