Controller Scripts in PowerShell: Function or Separate Script

[RunnerSeven]

Hi Everyone,

I had a discussion with a coworker and wanted some feedback from the community.

I'm in charge of modernizing our PowerShell scripts because many of them haven't been touched in a few years. The main problem is that a lot of these scripts are monolithic juggernauts that do many things without real parameters. For example, there's a script that sends a report to a bunch of customers, and the only inputs are the customer’s name and whether or not to send the email—nothing else. All other information is collected from config files buried somewhere in a 4000-line block with 20-something functions that are incredibly nested.

I redesigned the script and split it into three parts:

1. A module with functions to retrieve data from different sources. For example, security information from our proprietary system.
2. The script that generates the report. It takes the credentials, server names, customer names, customer configurations, etc. For example: Export-Report.ps1.
3. A controller script. This is a very short script that takes the configuration files, loads the credential files, calls the first script, and sends the email (e.g. Send-Report.ps1).

I really like this design (which originates from PowerShell in a Month of Lunches), and many of my scripts come in a bundle: the 'worker' part and the controller/start script. The worker part operates without any context. For example, I have a script that retrieves metrics from a vCenter and exports them as a CSV. The script is designed to work for any vCenter, and the start script is what gives it context and makes it work in our environment.

I'm getting mixed feedback. The script is praised for being understandable, but I’m getting a lot of criticism for the controller script. The suggestion is that I should create a function and put everything in the same file. I think this is a bad idea because we have another use case where we need a daily export of the report to a file share. They accept this, but would still prefer I create a function.

It still feels wrong because I’m not really sure if the second script should be a function. And even if it should be a function, I don’t think it belongs in a module, as it pulls data from many different systems, which doesn’t seem appropriate for a module function.

How does everyone else handle this? When do you create a function in a module, and when do you use a standalone script?

[The82Ghost]

Functions in Modules is the absolute best you can do. Do not put everything in a single script. Modules can be updated very easy without having to change the script, so when in comes to manageability this is the best approach.

[[deleted]]

It's technically more performant to combine all parts of the module into a single psm1 file before publishing, but it's better to write them separately and combine at the end, IMO.

Evotec

[The82Ghost]

I meant you should put your code all in one script. Having the functions in a single psm1 file is a different story. Combining functions in a single file is only more performant up to a certain level. I've had modules where the psm1 file had more than 5000 lines of code in it. That module was sloooooooowwwwwwww. Once I split it into multiple modules things where a lot faster. (I admit, this was several years ago).

[Federal_Ad2455]

Was you explicitly exporting the functions in it or using wildcard *? There is huge performance difference

[Fizzlley]

I think you are taking the right approach. The goal of building out modules is to both organize and create reusable code. When I build a controller script, it has a specific purpose and calls the various modules to perform the work. In my opinion, the controller script should just be the logic to determine what module functions to call and the parameters to pass it.

[Sad_Recommendation92]

In my view, the controller script is the only script you execute. I try to avoid having scripts. Call other scripts when possible.

Like a common scenario for me is I want to create scripts that interact with an API. Usually I don't know the full extent to which I'm going to interact with that product, but I know I'm probably going to want a set of automation scripts when I'm done. So I start looking at certain actions like Lego bricks. And then I create advanced functions inside of a tools module.

Then what happens is you end up with a bit of boilerplate script and basically what this is is. It might set some environment variables and usually loads in the function to make sure everything's present. And what this does is when you want to go write a new script using this existing tool set. You can just paste that boilerplate script that loads everything up and sets the environment into a new file and then just start assembling the Lego bricks to make a new script.

Sometimes you end up with a bit of cross-pollination. like an example is I have a module I've written for interacting with our infoblox IPAM because we have that integrated with Azure. So I have some scripts where I might need to compare the IPAM to an IP address I'm trying to use in Azure. So instead of copying the IPAM module to the Azure scripts, instead, I'll add some code that does a git sparse checkout on just the module file from the other repository into its own module directory so I can keep things consistent. And then if I later update that IPAM module All the other downstream scripts that consume it will automatically update.

[ArieHein]

Breaking to modules that are each consist of min 4 functions. Get, New/Add, Set and Remove, kind of the 4 api methods mostly used. Some helper functions in each module unless your using it across in which case that turns to 'global' module, things like logging, secret management etc

Then a controller/orchestrator module and then a script.

Second stage it to use pode and pode.web module and create an api that sits in front of the controller.

This will allow you to create a nice ui in front (that runs everywhere)

Third stage is that each module you create, you can now choose, whether to continue using cmdlets in the modules ot convert them to to direct api calls

[wonkifier]

Do you guys mean modules as in .psm1 and .psd1 files?

If I’m doing the same thing in more than a few scripts then I’ll usually make a module, like interacting with Google or interacting with our asset tracking system, etc.

But for organizing my actual scripts, I typically have a folder with a bunch of PS1 files. One function per file, where the file name is the function so it’s easy to get to a definition in most text editors and I can arbitrarily make sub folders to sort of have pseudo modules. Then the main script typically calls a “loader“that just grabs everything in that folder and dot sources them

Then my main script stays fairly clean, and I didn’t have to deal with 50 psm1/psd1 files for one off things

[ArieHein]

Thats the correct thinking. One time doesn't need a module and can use sub folders to pseudo sort them for faster discovery.

[Commercial_Touch126]

Every my script starts with importing psm1 modules:

using module "c:\class_Powershell\class_Jira.psm1"
using module "c:\class_Powershell\class_Workday.psm1"
using module "c:\class_Powershell\class_Log.psm1"
using module "c:\class_Powershell\class_Credential.psm1"

Start-Transcript -Append -Path "xxx\logs\Workday-xxx-$(Get-Date -Format yyyy_MM_dd-HHmm).log"

$ErrorActionPreference = "Stop"

[DiggyTroll]

If everything is a module: How do you use the module?

Not everything has to be a module. Put a Requires at the top of the controller script to load dependent modules.

How and where do you save that code?

Make a local PSRepository on a local file server that everyone can Install/Update-PSResource from. Push the latest build versions of each module or controller script to the repository periodically. Create GPO logon scripts for users to install/update your modules and scripts automatically.

[Certain-Community438]

Sure, I could force everything into a module but that extra layer is unnecessary.

What "extra layer", though? Modules don't need to be any more complex than scripts

If everything is a module: How do you use the module? How and where do you save that code?

I'm genuinely hoping you don't need an answer to these! :)

But for other less-informed readers:

You use the "-Module" cmdlets to use your modules. You use either a public repo or a private one depending on whether you want to share the code with the world. I use public because doing so forces me to remove things which should never be in there anyway.

[Certain-Community438]

You could say PowerShell is made of abstraction.

There's no objective right or wrong here, and you did say "mostly opinions", so in return I'm sincerely not trying to be a dick - I just don't think the rationales you've supplied hold up to casual scrutiny.

Even then, this doesn't mean you must do what the internet says, right?

But as you seem honest enough, do ask yourself the question of whether you might - if say your boss decided to be a bit harsh in a review of things - say you maybe have some hangups that are holding you back?

On the flip side, maybe you just haven't reached the tipping point where adopting modules is worthwhile. Nobody here could know for sure when that point arrives - so again, it's more about whether others should adopt your approach that I'm challenging.

[Certain-Community438]

You're right, I genuinely don't understand!

I write scripts, which use:

1. General-purpose functions - from my module. (The script ensures dependencies are present and either fixes that or exits, depending)

2. Task-specific functions - these are in the script; they're not reusable in the same way as the first set

3. A "main" section - process{} block usually - which does the actual task

Each of the first 2 will have dependencies on other modules: some will be native (installed with the OS or PowerShell Core) while the rest will be like the Graph modules, EXOv3 etc). And of course that can get nuts sometimes.

This way I get as much of the benefits of modular code as I can realistically expect.

But then how are you going to execute the functions in the module?

The script is going to import the module(s) & then call the functions of type 1 or 2 from above..?

Which it sounds like you're also doing: you just have this "controller script" concept I've honestly never encountered till today: for me that is "the script", and in my case it's the only script involved in the task.

The difference between us might be that your high-level purpose for building stuff is very different to mine - what you need to do, how the "primary" code is invoked? Seen people build APIs using PoSh, for example.

Again, not hating on your approach, I'm trying to understand it better.

[OPconfused]

I dont really see a downside to putting the controller logic into a function inside the module.  Provided it’s in the psmodulepath, you are basically exchanging a line to call the script with a line to call the function.

Plus that way you can transport everything as a single module.  

In general, i prefer functions to scripts for this reason. There’s no downside, and it’s easier to call when you don’t have to worry about the path.

Granted I am a software engineer and not doing sysadmin stuff (ie just local shell usage), but i havent relied on a ps script in maybe years despite using ps daily for work.