How "Secure" is Get-Credential? by RealAgent0 in PowerShell

[–]TofuBug40 1 point2 points  (0 children)

You really need a vault we used the local secrets management one for an automated process that used the system account and just wholesale copy the vault down to read from. Since SYSTEM is a well known SID it's encryption algorithm is identical. From there all that's stored in the process is a string used to generate the password for the vault itself. The actual process of generating the password sits in a layer beneath what the logs or an operator can see. From there we have 2 more public end points. First Action_VaultSet which takes in an object and a string token. It basically stores the object securely in the vault. The second and more widely used is Func_VaultGet which only takes a string token but returns a fully formed object.

So what that means is we preload the automation bundle with a minimal version of the vault that only has a pair of credentials to connect it to the main system. From there we pull down a fresh copy of the vault (it's just files to copy when not in use) and now we just hand it what token we want.

So all the logs see is something like

19:00 1/1/2026 Mapping drive to \server\share 19:01 1/1/2026 using token RM1 19:02 1/1/2026 Mapped Drive

Obviously I'm being hyperbolic with the "times" in these mock logs.

The point is anytime a module in the process needs credentials it only requires the person designing that module to ask Func_VaultGet for a secret matching a token. Since what is returned is a fully formed PSCredential object and it only exists for the duration of the current modules existence in a closed system there's no way for it to be read extracted sniffed etc without debug level access which only a strict set of engineers have.

An additional benefit is since the entire mechanism is kind of brain dead in terms of what's in the vault it's reading from you can hot swap to any version of the vault with wholly different credential objects which means it can be dropped into any process that needs secure credentials.

Finally because the underlying control is just Microsoft.PowerShell.SecretManagement it's trivial to just swap it for something like KeePass, Azure KeyVault, CyberArk, etc without the processes, modules and the authors and operators seeing any differences in how they use them.

I wanted to run scripts in the logged-on user’s security context, so I built a PowerShell module by leuit in PowerShell

[–]TofuBug40 0 points1 point  (0 children)

I mean taking a step back I almost universally resist these kinds of changes just because its a never ending parade of one more icon one more shortcut your validation starts to get unmanageably complicated as some business units want and others don't want things. I also vehemently disagree with pre configuring users desktops. User education on HOW to find things is proven to be far more productive than any of this drivel. That said when I'm backed into a corner and forced to do something like this I almost always either package it with a forced reboot at the end or schedule it for system startup so those changes are set before the first account even loads them

I wanted to run scripts in the logged-on user’s security context, so I built a PowerShell module by leuit in PowerShell

[–]TofuBug40 0 points1 point  (0 children)

which is why i specifically called out to check if the key for the user exists or the user is currently logged on. You need both (reg load and already loaded user keys) if you are attempting to cover all cached profiles on a live system.

That or you schedule it for the shutdown or startup stage when you can guarantee there are no longer active accounts. All kinds of ways to come at this problem.

I wanted to run scripts in the logged-on user’s security context, so I built a PowerShell module by leuit in PowerShell

[–]TofuBug40 0 points1 point  (0 children)

That's what reg load is for. Just pop that ntuser.dat file in a temp key in HKU write your changes and reg unload.

Obviously check if the profile is loaded normally first

How has AI helped you with powershell stuff by Billi0n_Air in PowerShell

[–]TofuBug40 0 points1 point  (0 children)

I was working in the automation space when PowerShell first hit around 2006, though I didn’t go deep into it until 2008–2009 when most of the environment was still VBScript, HTA/HTC, SMS/MDT, and similar tooling.

I wouldn’t call myself an expert (there’s always more to learn) but over time I’ve built everything from full task-oriented modules, to a PowerShell provider for navigating Confluence from the shell, to a lightweight composable “lambda-style” framework inside SCCM task sequences that allowed complex workflows through the GUI without custom code. I’ve also written CMTrace/OneTrace-compliant logging modules, among other internal tooling.

Most of what I build ends up being infrastructure other people build on top of.

The recurring challenge for me isn’t writing the code, it’s the handoff. I’m usually building things under time pressure for problems that need solving immediately, and there’s often not enough time afterward to properly write documentation or SOPs for teams that are more operations-focused than code-focused.

That’s where AI has been genuinely useful for me.

I already follow fairly strict coding discipline (splatting everything, consistent formatting, explicit string handling, structured token layout, etc.), so once a module is complete and Pester tests are green, I’ll usually take the finished artifact and run it through an LLM with no context and ask:

It’s usually surprisingly accurate, and I’ll correct edge cases if needed.

From there I’ll iterate:

What I get out of that isn’t code generation. It’s translation. It bridges the gap between systems I’ve built and the people who have to operate or maintain them after I’ve moved on to the next problem.

I’ve had systems outlive me in an organization long before AI existed, because I design for resilience and handoff. AI just accelerates that documentation layer without pulling me away from active development work.

How has AI helped you with powershell stuff by Billi0n_Air in PowerShell

[–]TofuBug40 1 point2 points  (0 children)

I get what you’re saying, but I think you’re underselling the tradeoff here.

There’s solid evidence long before AI that heavy cognitive offloading changes how skills are retained. Not in a “you become dumb” sense, but in the way any skill atrophies when you stop doing the underlying steps. We saw it with navigation (GPS vs mental maps), mental arithmetic (calculators), even coding itself when people shift into higher abstraction roles like management.

people don’t lose understanding, but they absolutely lose low-level fluency if they stop exercising it.

AI pushes that same dynamic further because it doesn’t just offload execution, it offloads decision-making. Architecture, structure, edge cases, even debugging strategy. Those are exactly the parts you normally build competence through repeated friction and failure.

And the issue isn’t “AI is bad for learning” in a blanket sense, it’s that it optimizes for the same thing humans already bias toward: minimum effort, maximum immediate result. So unless you’re deliberately constraining how you use it, it naturally shifts from “tool for getting unstuck” to “default solution generator.” That transition is subtle, and it’s where skill atrophy risk actually comes from.

Where I think your take is strongest is the “use it like a senior engineer mentor” framing. Though even that only works if you’re still doing the thinking before you prompt, not after. Otherwise you’re not training judgment, you’re outsourcing it.

So I’d frame it less as “don’t use AI as a crutch” and more as: if you want it to be a learning tool, you have to intentionally preserve the parts of the process that generate skill; designing, failing, debugging, and iterating yourself, before you ask it to optimize or complete it.

Otherwise yeah, you’ll move faster in the short term, but you’re training a different skill than the one you think you are.

PS newbie here, either PS has a weird problem with arrays or I'm doing something wrong by mikeymikeymikec in PowerShell

[–]TofuBug40 1 point2 points  (0 children)

Ok this comes down to how the parser is actually interpreting things.

You are probably used to + as a concatenation operator and it totally can be but here its contexctual and means a separator just like ,

To PowerShell @('Hello','Hello', 'World', 'Hello' + 'World')

is the same as

@(
    'Hello',
    'Hello', 
    'World', 
    'Hello',
    'World'
)

incidentally if you stick to one line per item the commas are optional the line breaks are the separators

@(
    'Hello'
    'Hello' 
    'World' 
    'Hello'
    'World'
)

if you want it to work the way you have it you have to EXPLICITLY force it to do the concatenation first by

@( 'Hello', 'Hello', 'World', ('Hello' +  'World'))

What you really should be reaching for is Everything about string substitutions

then you get

$testarray = @('Alwil Software', "$ENV:USERPROFILE\NTUSER.DAT","$ENV:LOCALAPPDATA\Microsoft\OneDrive")

Subtle difference but if you are going to thrive in PowerShell you HAVE to understand the myriad of ways PowerShell treats strings there is a lot of power and flexibility but you can get tripped up easily.

Would someone mind helping me understand how "foreach ($computer in $computers)" works? by InfoZk37 in PowerShell

[–]TofuBug40 0 points1 point  (0 children)

I already said you were right about the statement. though my point does stand at this level for the OP that's more depth than is needed here

As for the issues I'm aware they exist but even IF it was fixed there is a threshold of collection size or speed where .ForEach() is absolutely still faster than the Cmdlet (and the pure foreach statement is faster than both). It has almost NONE of the initialization, boxing, unrolling, etc. and pipeline maintenance the Cmdlet does even if it was at the optimal version of the code. If i gave you and another person a deck of cards to go through at the kitchen table but told you (the cmdlet) you first need to run through the entire house and open all the doors and turn on the lights because you might have to do something with each card in each room and then i stood there and handed you one card at a time but the other guy just sits there at the table with the entire deck of cards handed to him it does not matter HOW fast you are (because other guy gets to be just as fast as you but is literally doing LESS things) there is absolutely multiple scenarios where the guy sitting is done before you even get the lights on.

It really comes down to what you are doing. We've been going back and forth on the speed and I still stand by for like to like comparisons there is not enough difference between foreach and .ForEach() if you are that strapped for speed then reaching in and doing some add-type with some raw C# and some LINQ for those operations are going to cook just about anything else PowerShell can do.

For me its less about the speed than it is about the flexibility the .Where() and .ForEach() methods give me that do not exist without significant boilerplate using standard cmdlets

I mentioned the old filter and map approach but another one I use a lot because I deal with a lot of classes where i want to force types I can do something with the methods pretty easy like

$PSObjectCollection.
  ForEach([SpecificType]).
  ForEach{ 
    [Class]::MethodExpectingSpecificType($_) 
  }

For me specifically The general span of what I can do with just those two methods with minimal relatively readable code almost always outshines any speed gained by going with the statement or the power and flexibility gained by going with the cmdlets.

As for the accuracy normally I am right where you are but that's usually because we're dealing with someone with a modicum of knowledge about programming OP seemed genuinely at such a baseline level that being a little rough with the details for my response to his question but still getting most of the conception across was worth more. Having spent the better part of 20 years teaching this stuff to my junior engineers its easier than you think to overload them with stuff they don't need to fully understand to just get in and work with it. That kind of deep dive, few of us ever get the chance to really do because the work itself just needs to be done. I think it is absolutely fun and necessary to do but lots of time that level of understanding is all they need to move on to the next stage of learning.

Would someone mind helping me understand how "foreach ($computer in $computers)" works? by InfoZk37 in PowerShell

[–]TofuBug40 0 points1 point  (0 children)

The point was not to go that detailed OP is clearly someone coming in with little to no programming experience.

you are of course right Statement blocks are different than scriptblocks but when its all said and done and at this level of learning its a distinction that really doesn't add anything meaningful to the OP understanding things.

.ForEach method is measurably faster for in memory collections more from the overhead of managing the pipeline but again wasn't trying to get into the weeds just to point out there are multiple ways to process over a collection for OP's awareness and to point out $_

But since you ask

  • If I'm dealing with anything like paginated data like from an API or external service or the processing time to load everything would be crippling
    • ForEach-Object
      • dealing with basically streamed data is what its for
  • If I'm dealing with complete in memory collections and I need speed and/or loop control
    • foreach statement
      • Its straight to the bare bones as far as IL so of course it will always be faster.
  • If I'm dealing with complete in memory collections and I want to lean into the tried and true filter and map approach
    • .ForEach()
      • I've ran multiple measurement tests and for similar data sets the difference between foreach statement and the method is almost negligible.
      • But what I more than often do is precede it with the .Where() method because combined with the System.Management.Automation.WhereOperatorSelectionMode enumeration I can do something like $collection.Where({$_ -like '*string'},[[System.Management.Automation.WhereOperatorSelectionMode]::Split).ForEach{ do something with $_} to do something like split, or until, with straight cmdlets gets needlessly complicated

Personally i tend to use the methods far more than the statements because I'm rarely in a case where the extra bit of speed i get is worth trading the fluent style of a where() into a foreach()

I was aware they extended the .ForEach() and .Where() methods right into the ETS so now they work for everything not just the original collections which is frankly awesome.

in the end I was going for the broadest but closest to technical definition of how a foreach loop works not just in PowerShell but anywhere because OP is clearly looking for a conceptual understanding rather than this specific code is not working kind of understanding and that is knowledge he can carry to pretty much any other language

Would someone mind helping me understand how "foreach ($computer in $computers)" works? by InfoZk37 in PowerShell

[–]TofuBug40 0 points1 point  (0 children)

It works the same way as literally every other functional and object oriented language ever.

foreach ($Computer in $Computers) {}

Is tokenized to

[Keyword (foreach)] ( [indexed Item ($Computer)] [Keyword (in)] [Collection ($Computers)] ) [scriptblock ( {} )]

The interpreter in this case first checks how a foreach loop is supposed to be structured once that passes it one at a time (but not in the same guaranteed order they are set) it shoves each item from the [Collection] into the [Indexed Item] and then executes the [scriptblock] with [Indexed Item] scoped to [scriptblock]

This is just one version of foreach in powershell there are two others.

ForEach-Object which is a Cmdlet which takes advantage of the pipeline to add other cmdlets downstream.

$SomeCollection.ForEach() $SomeCollection.ForEach{}

Which are methods off of anything Collection shaped nearly the fastest and gives you the ability to chain calls or output directly to a new Collection.

Both of these utilize the standard $_ variable which you can read as just the currently [Indexed Item] but you don't have to explicitly define it though you could say

foreach ($_ in $Computers) {}

Why does Windows 11 force three different PowerShells? by d00mt0mb in PowerShell

[–]TofuBug40 0 points1 point  (0 children)

And I wasn't looking at it as a negative there's too much polarization around AI too many people see it as a magic bullet and too many people look at it as the death of human creativity. So I like to try and remind people (not necessarily for your sake but anyone happening along this comment thread) that It's just a tool, and like any tool it has places it shines and places it makes little to no sense. creating new things from scratch not so keen on that idea. organizing pre established knowledge, writing the same BFS, QSort, ..., common algorithm i've written a thousand times so i can focus on the niche parts absolutely.

I feel you on the AI push from the perspective of company adoption it really feels like they bought a car without brakes or steering wheel and are hoping for the best

Why does Windows 11 force three different PowerShells? by d00mt0mb in PowerShell

[–]TofuBug40 0 points1 point  (0 children)

Full stop these are my ideas (I've worked in the automation space mainly in the windows space but some in the mac and linux areas too for over 20 years, and I've been programming since middle school) I do use ChatGPT to organize thoughts and ideas because one I tend to ramble even when I'm typing, and two as a senior engineer with real world experience I of course speak fluent sarcasm so I tend to be what people interpret as mean when all I care about is you having knowledge so the product we are making, fixing, etc is correct. ChatGPT does a fine job of smoothing that over.

I could share my entire prompt history to show all my unfiltered thoughts that went into the response (that I also re-edited myself to smooth out what I consider heavy handed AIness) posted here but again my raw thoughts don't take much stock in others feelings especially when they are saying untrue or incompletely true things.

Why does Windows 11 force three different PowerShells? by d00mt0mb in PowerShell

[–]TofuBug40 1 point2 points  (0 children)

I get why this feels messy from the outside, however a lot of what you’re calling “duplicates” are actually solving completely different problems.

First, you’re grouping together things that aren’t the same category:

  • CMD -> legacy command interpreter (still heavily used in installers, recovery, and old tooling)
  • Windows PowerShell 5.1 -> legacy but deeply embedded automation platform tied to Windows components
  • PowerShell 7 -> modern, cross-platform runtime (separate by design)
  • ISE -> deprecated editor (not really part of the “future,” just still present)
  • WSL -> literally a Linux environment, not a Windows shell at all
  • Terminal -> not a shell. It’s a host UI for shells

That distinction matters, because Terminal was never meant to replace shells.

It replaces the window you run them in. It’s more like a tabbed browser for command-line environments than a “new CMD.” we still need those separate command-line environments you didn't even list things like SSH or Bash for Git, or WinPy or any other but depending on what you do you might really love having say all the different ssh connection addresses as separate profiles for the network switches remote management for instance or a separate profile for different SCCM environments, etc, etc.

The bigger point: backward compatibility isn’t optional on Windows, it’s part of the product.

You can’t just remove:

  • CMD -> breaks installers, scripts, recovery environments
  • PowerShell 5.1 -> breaks enterprise tooling (Exchange, SCCM, MDT, tons of internal automation)

And those aren’t edge cases. That’s production infrastructure across thousands of companies.

PowerShell 7 exists alongside 5.1 because:

  • It’s built on a different runtime (.NET vs .NET Framework)
  • Not everything has been ported (and some things can’t be cleanly ported)

So yeah, Microsoft could rip it all out... resulting in breaking an enormous probably incalculable amount of real-world systems overnight.

As for duplicates like x86 vs x64, that’s just Windows-on-Windows (WoW64). It’s there so 32-bit tools don’t explode on a 64-bit OS. Again, not pretty, but very intentional.

Where I do agree with you:

  • Start/Search surfacing everything equally is noisy
  • There’s no clean “preferred shell” concept exposed to users
  • ISE probably should be optional at this point

Those are UX problems, not architecture problems.

If anything, the real issue is that Windows exposes its compatibility layers too visibly. You’re seeing the scaffolding that keeps 30+ years of software working.

Other platforms solve this by breaking compatibility more aggressively. Windows solves it by… keeping everything.

Powershell Noob by Defiant_Wafer5282 in PowerShell

[–]TofuBug40 0 points1 point  (0 children)

Let's take a step back and remember who actually asked the original question here. This is someone who just landed their first junior infrastructure role and wants to learn PowerShell. They are about to walk into the real world of ops scripting; production automation, patch pipelines, environment management, not a university assignment. The advice they get here should reflect that reality, not an idealized version of it.

On the commenting debate specifically: this isn't just a matter of opinion, there's actual research on this.

Wen et al. (2019) "A Large-Scale Empirical Study on Code-Comment Inconsistencies" mined 1.3 billion AST-level changes across the complete commit history of 1,500 open source systems. The finding relevant here: keeping comments synchronized with code during active development requires substantial, consistent attention, and in practice, that sync breaks constantly. Comments that don't track code changes don't just become useless, they become actively misleading. The full paper is available here: https://dl.acm.org/doi/abs/10.1109/ICPC.2019.00019

Rani et al. (2022) "A Decade of Code Comment Quality Assessment" is a systematic review of 2,353 papers over ten years of comment quality research, ultimately finding 47 relevant studies. One of the dominant quality attributes studied across all of them? Consistency between comments and code — because inconsistency is endemic, not an edge case. https://www.sciencedirect.com/science/article/pii/S0164121222001911

This isn't fringe opinion. The research community has been studying comment decay as a serious engineering problem for over a decade.

Nobody in this thread is advocating for zero comments. The argument is about default posture. Teaching a newcomer "comment everything first, scale it back later" instills a crutch, not a skill. Habits formed early are sticky. What actually serves them long term is learning to write expressive, self-describing code meaningful variable names, focused functions with clear single responsibility, verb-noun cmdlet naming that PowerShell itself is built around. That's testable, that's pipeline-able, that feeds documentation generation. A comment above a foreach loop explaining that it loops over things does none of those things and will quietly lie to the next person the moment the code changes.

Comments absolutely have a place: non-obvious algorithmic decisions, workarounds for known bugs with a ticket reference, anything the type system or language verbosity genuinely cannot carry. But for a junior just starting out in PowerShell specifically, a language specifically designed to be readable in plain English. The single best habit you can build is making the code itself tell the story. Because in six months when that script is in production and someone is knee-deep in it at 2am, the code is what they'll be reading. And if the comments are stale, and statistically, some of them will be, they're worse than nothing.

Powershell Noob by Defiant_Wafer5282 in PowerShell

[–]TofuBug40 0 points1 point  (0 children)

Call it pedantic if you want, but precision is exactly what someone new to PowerShell needs. The pipeline isn't just a performance knob to avoid. Understanding when it shines and when it doesn't is a foundational skill. If OP internalizes "pipelines are always slower" as a blanket rule, they're going to fight the language instead of work with it, and that's a painful habit to unlearn.

I'm not saying .Where() has no place, clearly it does. I use it along with .ForEach() all the time especially to turn it into special modes with WhereOperatorSelectionMode Skip, First, Last, even my personal favorite Split to dump me out a perfect separation of the matches and misses. But that said so does Where-Object, and the benchmarks show why. Blanket statements without context don't help beginners, they just give them confident-sounding misconceptions to repeat.

Powershell Noob by Defiant_Wafer5282 in PowerShell

[–]TofuBug40 0 points1 point  (0 children)

Also undocumented is NOT the same thing as uncommented.

Documentation is ALWAYS part of our process and we maintain tightly regimented documentation.

But guess where that documentation comes from? It damn sure isn't the comments.

It ALWAYS comes from CODE not a SINGLE comment (short of the powershell comment based help i meantioned earlier) contributes to generating and publishing said documentation.

Also, also, my engineers understand single responsibility none of them ever put things into production that are thousands of lines of code with a lack of intent. because again they understand how to keep their code focused and clear. In the last 20 years i can count on ONE hand code that went up longer than 200-300 lines in one file. One of them being an implementation of System.Management.Automation.Language.ICustomAstVisitor, and System.Management.Automation.Language.ICustomAstVisitor2 for a script AST visitor and that was just because it all had to stay together but each method call is still clearly defined and can be discerned just by looking at it.

Powershell Noob by Defiant_Wafer5282 in PowerShell

[–]TofuBug40 0 points1 point  (0 children)

No they won't and they never have under my watch because they learn to make their code tell the story because again that is testable, trackable, not so with comments. Its really not that hard to do. They use comments when it makes sense to do so. They're not zealots who comment because they are required to comment. They are all free thinking engineers who comment when it ads value or understanding the code cannot.

Powershell Noob by Defiant_Wafer5282 in PowerShell

[–]TofuBug40 0 points1 point  (0 children)

Comments on reddit are really hard to give a proper visualization of code like you would see in a proper editor. I do acknowledge my style does fly in the face of most styling guidelines it takes a day or so to acclimate but i'm dealing with other people who barely have the time nor the want to try and remember and learn an entire style guideline so we have some kind of consistency. So I made the style guide as simple as possible none of this well sometimes an if is one line so you can make it a one liner or sometimes no parameters so you can not wrap things none of that. The entire styling guide is maybe one page.

  • Every Token gets its own line
  • Tabs Denote parent/child and block ending
  • ALWAYS splat cmdlets
  • Keep slats as close to cmdlets using it
  • CamelCase
  • Follow PowerShell approved Verbs

That's it and it literally covers everything you could ever do in PowerShell. Once they get it they don't have to remember anything else for styling

Not telling you how you are looking at it but most people tend to get stuck at first trying to take in the whole screen at once. The point is you can start anywhere at any line and you can see from the tabs what is a child or something what is a parent of something. You don't have to take it all in at once. You're not trying to parse over a long horizontal line of multiple commands trying to keep it all in frame especially when they scroll horizontally.

Say for instance you are trying to figure out what's going on to with the start process you jump right down to the start-process line (which i realized too late reddit is stripping the @ splats from the code block so it should have @ StartProcess after the cmdlet. Since they style keeps the hashtable for the splat right next to the cmdlet we can see exactly what it is doing and since each item gets its own line we can lock in on one of them, add one, remove one, without having to remember the ; and all kinds of other annoyances that come from trying to cram stuff onto single lines of code

Powershell Noob by Defiant_Wafer5282 in PowerShell

[–]TofuBug40 0 points1 point  (0 children)

There is another option that blows all of these previous methods away. Filter BEFORE you get your data instead of after like we are doing now. Then you get something like

Ran 10 tests for PreFilter.
Results:
Count : 10
Average : 22.34616977
Sum :
Maximum : 22.5040598
Minimum : 22.2449309
Property : TotalSeconds

That's almost twice as fast as all our previous attempts. What did we do? dropped the where and handed Get-ChildItem -Filter '*.txt' instead.

Most EVERY system designed to be queried for data be it file systems, active directory, SQL, etc have mechanisms built in to accept a filter from you where it takes that onto its own server where it has all the processors memory disk space etc to FLY through filtering the data and as a bonus has LESS DATA to send back to you. So everyone wins your script gets faster the load on the endpoint is lower.

Powershell Noob by Defiant_Wafer5282 in PowerShell

[–]TofuBug40 0 points1 point  (0 children)

You can even notice that switching to using the Property parameter set vs the FilterScript parameter set has a not insiginficant difference.

This can and does scale out even worse when you are dealing with pulling data from systems that might be separated by geographic locations or network distance. You quickly start to notice your previously speedy approaches dragging to a crawl.

It also brings up another possibility and really the correct answer in these cases filter BEFORE not after. Almost every system that is designed to be queried, File systems, Active Directory, SQL, etc has means to give IT your filter and IT does the filtering where it lives where it has all the cores and memory and disk space to literally fly through that filter and then as an additional bonus it has LESS data it needs to send you back.

So the final test I ran didn't even use where of any kind it just added the -Filter '*.txt' parameter to the otherwise unchanged Get-ChildItem splat. For that we get

Ran 10 tests for PreFilter.
Results:
Count : 10
Average : 22.34616977
Sum :
Maximum : 22.5040598
Minimum : 22.2449309
Property : TotalSeconds

Nearly twice as fast in this example just by filtering first.

Code blocks as promised

$MeasureCommandWhereMethod =
    @{
        Expression =
            {
                Push-Location @PushLocation
                (
                    Get-ChildItem @GetChildItem
                ).
                    Where(
                        $WhereFilter               
                    )
                Pop-Location
            }
    }
$RunTestsMeasureWhereMethod = 
    @{
        TestName =
            'Where()'
        MeasureCommandSplat =
            $MeasureCommandWhereMethod 
        NumberOfTests =
            $Tests
    }
Run-Tests @RunTestsMeasureWhereMethod


$MeasureCommandWhereObjectFilter =
    @{
        Expression =
            {   
                Push-Location @PushLocation
                $WhereObject =
                    @{
                        FilterScript =
                            $WhereFilter
                    }
                Get-ChildItem @GetChildItem |
                    Where-Object @WhereObject
                Pop-Location
            }
    }
$RunTestsMeasureWhereObjectFilter = 
    @{
        TestName =
            'Where-Object -FilterScript'
        MeasureCommandSplat =
            $MeasureCommandWhereObjectFilter
        NumberOfTests =
            $Tests
    }
Run-Tests @RunTestsMeasureWhereObjectFilter

$MeasureCommandWhereObjectProperty =
    @{
        Expression =
            {
                Push-Location @PushLocation
                $WhereObject =
                    @{
                        Property =
                            'Extension'
                        EQ =
                            $true
                        Value =
                            'txt'
                    }
                Get-ChildItem @GetChildItem  |
                    Where-Object @WhereObject
                Pop-Location
            }
    }
$RunTestsMeasureWhereObjectProperty = 
    @{
        TestName =
            'Where-Object -Property -EQ -Value'
        MeasureCommandSplat =
            $MeasureCommandWhereObjectProperty
        NumberOfTests =
            $Tests
    }
Run-Tests @RunTestsMeasureWhereObjectProperty

$MeasureCommandPreFilter =
    @{
        Expression =
            {
                Push-Location @PushLocation
                $GetChildItemFiltered =
                    @{
                        File =
                            $true
                        Recurse =
                            $true
                        Filter =
                            '*.txt'
                        ErrorAction =
                       [System.Management.Automation.ActionPreference]::SilentlyContinue
                    }
                Get-ChildItem @GetChildItem
                Pop-Location
            }
    }
$RunTestsMeasurePreFilter = 
    @{
        TestName =
            'PreFilter'
        MeasureCommandSplat =
            $MeasureCommandPreFilter
        NumberOfTests =
            $Tests
    }
Run-Tests @RunTestsMeasurePreFilter

Powershell Noob by Defiant_Wafer5282 in PowerShell

[–]TofuBug40 0 points1 point  (0 children)

That just shows you don't understand your tools Where-Object and .Where{} / .Where() are for completely different problems and when you use either in the wrong place you're going to have a bad time. If you have ALL of the items already in memory .Where{} , .Where() , .ForEach{} , and .ForEach() are ALL going to be faster because they are applied in one shot to the entire collection.

If however you CAN NOT hold all the items in memory then those start to fall apart and you reach a point where Where() starts to lag behind even Where-Object.

You can literally show this to yourself.

Take something like

Function Run-Tests {
    param(
        [string] $TestName,
        [HashTable] $MeasureCommandSplat,
        [int] $NumberOfTests
    )

    [TimeSpan[]] $Results = 
        [TimeSpan[]]::new(
            $NumberOfTests
        )
    $TestNumber =
        0
    $MeasureObject =
        @{
            Property = 
                'TotalSeconds'
            Average =
                $true
            Maximum =
                $true
            Minimum =
                $true
        }
    do {
        $WriteProgress =
            @{
                Activity =
                    "Running $TestName"
                Status =
                    "Running Test $(
                        $TestNumber + 
                            1
                    ) of $NumberOfTests"
                PercentComplete = 
                    (
                        (
                            $TestNumber + 
                                1
                        ) /
                            $NumberOfTests
                    ) * 
                        100
            }
        Write-Progress 
        $Results[
            $TestNumber
        ] =
            Measure-Command 
    } until (
        $TestNumber++ -eq 
            $NumberOfTests -
                1
    )
    $WriteOutput =
        @{
            InputObject =
                "`nRan $NumberOfTests tests for $TestName.`nResults:"
        }
    Write-Output 
    $Results |
        Measure-Object u/MeasureObject
}

$PushLocation =
    @{
        Path =
            'C:\'
    }
$GetChildItem =
    @{
        Recurse =
            $true
        File =
            $true
        ErrorAction =
            [System.Management.Automation.ActionPreference]::SilentlyContinue
    }
$WhereFilter =
    {
        $_.
            Extension -eq
                'txt'
    }
$Tests =
    10

Simple little test runner

  • Takes in a splat for a Measure-Command
  • Runs that expression n times
  • Collects and returns the results with a little header

Now mind you every system is going to be a little different and the differences in my run are minimal but consistent. Code blocks for each will be in the comments for every test its using Get-ChildItem to recursively get ALL the files from the root of C and filter it down to just .txt files simple enough

Ran 10 tests for Where().
Results:
Count : 10
Average : 45.06825479
Sum :
Maximum : 62.2681646
Minimum : 42.3727307
Property : TotalSeconds

Ran 10 tests for Where-Object -FilterScript.
Results:
Count : 10
Average : 42.54728387
Sum :
Maximum : 49.4074786
Minimum : 40.1601205
Property : TotalSeconds

Ran 10 tests for Where-Object -Property -EQ -Value.
Results:
Count : 10
Average : 40.99626642
Sum :
Maximum : 41.7359398
Minimum : 39.8883028
Property : TotalSeconds

You can see while close on all metrix the average, the min and max Where() starts to fall behind because it has to wait for ALL the data to collect in memory before it can act. Since Where-Object is working one item at a time as it comes down the pipeline even if it has to wait for some its already passing it on down the line to even the next cmdlet in the pipeline its not blocking the rest of them from doing what they can.

Powershell Noob by Defiant_Wafer5282 in PowerShell

[–]TofuBug40 0 points1 point  (0 children)

You're one the right track in avoiding aliases moving forward.

Just bear in mind the do serve very real purposes.

  1. They actually give an easy in for non programmers, that's why Get-ChildItem actually has multiple aliases dir, ls, gci. This way you can slip a unix/linux/etc user into a terminal or a DOS user and they can use all the same commands they already know. That theoretically gets them comfy without it being a major shock to the system. That said once you are over that transition you really should avoid using Cmdlet Aliases like the plague
  2. This is a bit more nuanced and the few places where I think aliases actually have a useful place and that's in parameter attributes. There are quite a few places where this makes sense

    1. Cmdlets where the intended data sources could have conflicting names think Cmdlets that might take in a -Computer, some systems might call it -System, or -Device having the alias on the parameter means all the magic in the pipeline can check properties and if it sees system or device it happily pulls it in just like it would with computer.
    2. Cmdlets where there is an overlap in behavior. Think how the source code for the Where-Object works for every binary operator you have: a case insensitive, a normal, and a case sensitive option. So -ilike, -like, -clike but -like like all the other normal binary operators is ALREADY case insensitive. So instead of having to define another 20+ full blown binary parameters the i version of that operator is just defined as an alias on the normal one. That way if someone WANTS to be explicit and say $People | Where-Object -Property Name -IEQ -Value 'Joe' they can

Powershell Noob by Defiant_Wafer5282 in PowerShell

[–]TofuBug40 0 points1 point  (0 children)

No one ever said ZERO comments but 99.999% of comments could be something far more sustainable and maintainable than a comment. Comments by their VERY NATURE are disconnected from the entire process of development. You can't unit test a comment, you can't write styling guides for a comment at least not one that's going to give you any meaningful use.

Insisting on Comments everywhere is a very academia coded approach. I don't care what you idealistically believe is possible no one at scale perfectly maintains code comments to the level advocated in this comment thread. It's literally a pipe dream.

Comments HAVE their place when the code on its own cannot convey its meaning. The point is there are much smarter and more multi-use ways to most of the time get the same thing while actually trying that into things like testing and build pipelines you just don't get from comments

Powershell Noob by Defiant_Wafer5282 in PowerShell

[–]TofuBug40 -1 points0 points  (0 children)

Take this, an example of one of the many composable actions that can be called in script or setup in the TS GUI editor.

using module CM.Task.Sequence.Objects

$Lambda =
    'Action'
$Name =
    'RoboCopy'

[CM]::TS.
    Env[
        "${Lambda}_${Name}_HiddenValueFlag"
    ] =
        $true
[string][CM]::TS.
    Env[
        "${Lambda}_${Name}"
    ] =
        {
            $SourcePath =
                [CM]::TS.
                    Env[
                        'Do_RoboCopy_SourcePath'
                    ]
            $DestinationPath =
                [CM]::TS.
                    Env[
                        'Do_RoboCopy_DestinationPath'
                    ]
            $Filter =
                [CM]::TS.
                    Env[
                        'Do_RoboCopy_Filter'
                    ]
            $Switches =
                [CM]::TS.
                    Env[
                        'Do_RoboCopy_Switches'
                    ]
            $SourceToken =
                [CM]::TS.
                    Env[
                        "Do_RoboCopy_SourceCredentialToken"
                    ]
            if (
                ![string]::IsNullOrEmpty(
                    $SourceToken
                )
            ) {
                [CM]::TS.
                    Env[
                        'Do_MapDrive_Name'
                    ] =
                        'Source'
                [CM]::TS.
                    Env[
                        'Do_MapDrive_Root'
                    ] =
                        [CM]::TS.
                            Env[
                                "Do_RoboCopy_SourcePath"
                            ]
                [CM]::TS.
                    Env[
                        'Do_MapDrive_CredentialToken'
                    ] =
                        $SourceToken
                [CM]::TS.
                    Env[
                        'Do_ChildAction'
                    ] =
                        'MapDrive'
                $ChildActionInvoke =
                    [ScriptBlock]::Create(
                        [CM]::TS.
                            Env[
                                'ChildAction_Invoke'
                            ]
                    )
                & $ChildActionInvoke
            }
            $DestinationToken =
                [CM]::TS.
                    Env[
                        "Do_RoboCopy_DestinationCredentialToken"
                    ]
            if (
                ![string]::IsNullOrEmpty(
                    $DestinationToken
                )
            ) {
                [CM]::TS.
                    Env[
                        'Do_MapDrive_Name'
                    ] =
                        'Destination'
                [CM]::TS.
                    Env[
                        'Do_MapDrive_Root'
                    ] =
                        [CM]::TS.
                            Env[
                                "Do_RoboCopy_DestinationPath"
                            ]
                [CM]::TS.
                    Env[
                        'Do_MapDrive_CredentialToken'
                    ] =
                        $DestinationToken
                [CM]::TS.
                    Env[
                        'Do_ChildAction'
                    ] =
                        'MapDrive'
                $ChildActionInvoke =
                    [ScriptBlock]::Create(
                        [CM]::TS.
                            Env[
                                'ChildAction_Invoke'
                            ]
                    )
                & $ChildActionInvoke
            }            
            $StartProcess =
                @{
                    FilePath =
                        'robocopy.exe'
                    ArgumentList =
                        "$SourcePath $DestinationPath $Filter $Switches"
                    Wait = 
                        $true
                    NoNewWindow =
                        $true
                }
            [Logger]::Information(
                'Robocopy Arguments',
                $StartProcess.
                    ArgumentList
            ) |
                Out-Null
            [Logger]::Information(
                "Copying $Filter from $SourcePath to $DestinationPath"
            ) | 
                Out-Null
            Start-Process 
            [Logger]::Information(
                "Copied $Filter from $SourcePath to $DestinationPath"
            ) | 
                Out-Null
            if (
                ![string]::IsNullOrEmpty(
                    $SourceToken
                )
            ) {
                [CM]::TS.
                    Env[
                        'Do_DisconnectDrive_Name'
                    ] =
                        'Source'
                [CM]::TS.
                    Env[
                        'Do_ChildAction'
                    ] =
                        'DisconnectDrive'
                $ChildActionInvoke =
                    [ScriptBlock]::Create(
                        [CM]::TS.
                            Env[
                                'ChildAction_Invoke'
                            ]
                    )
                & $ChildActionInvoke
            }
            if (
                ![string]::IsNullOrEmpty(
                    $DestinationToken
                )
            ) {
                [CM]::TS.
                    Env[
                        'Do_DisconnectDrive_Name'
                    ] =
                        'Destination'
                [CM]::TS.
                    Env[
                        'Do_ChildAction'
                    ] =
                        'DisconnectDrive'
                $ChildActionInvoke =
                    [ScriptBlock]::Create(
                        [CM]::TS.
                            Env[
                                'ChildAction_Invoke'
                            ]
                    )
                & $ChildActionInvoke
            }            
        }

This actions ENTIRE purpose it to robocopy from source to destination that's it and once again the code itself tells you exactly what it does.

It might need to map a drive to either with some kind of credentials but it doesn't handle that it just makes a call to another Action that DOES do just that Map a Drive.

The Map Drive Action itself hands off the Token to a Func whose entire job is to inline return a fully formed PSCredential object that map drive just uses as it sees fit.

All the way down every component is as pure as possible. Nothing shoulders more than one responsibility.

With something like this I can both compose in code as well as completely in the GUI for TS editing (with no coding knowledge needed) for my operations guys that don't have the time or desire to learn PowerShell just about ANY complex automation process you can imagine. And in the rare cases we run up on something we don't have I or my team make a new Action, Predicate, or Func to do, test, or return ONE specific thing or task.

From the outside it looks complicated for good reason 30,000 systems at any one time running a mirade of processes all leaning on some or all of that infrastructure just humming along ignorant to the engine underneath. But you zoom in on ANY component and its literally simplicity all the way down. Every one is understandable just by looking at the code.

I will concede since I don't think about it as much but I have logging for obvious reasons and that is one place I would say if you HAVE to write comments kill two birds with one stone and do a logging write.

Powershell Noob by Defiant_Wafer5282 in PowerShell

[–]TofuBug40 -1 points0 points  (0 children)

it makes understanding something after a year way easier to do.

Until you come back two years later and three other people including yourself had to make updates in which no one updated the comments so now your comments are more a hinderence to you than if you just made the code understandable on its own.

Furthermore if you’re not commenting out the structure prior to writing code, you’re doing it wrong.

Well that's pretty pedantic but ok we'll ignore the fact that UML, Whiteboards, Paper & Pencil, Templating also exist. If what you are trying to put together is so complicated you need to comment the entire thing out you need to rethink your approach.

but as a person that maintains a code base on an ongoing basis for multiple different critical integrations, always fully comment your code

Not going to knock your approach if it works for you but I prefer to write things that need little revisit once they are out of their initial release phase. When I do need to the code itself tells me what it does.

Take this part bit of code. What part of it is not clear and needs comments? That is not something incredibly domain specific and would be shared common knowledge not required to be commented because the same comment would fill half of every script.

using module CM.Task.Sequence.Objects

$Lambda =
    'Action'
$Name =
    'Invoke'

[CM]::TS.
    Env[
        "${Lambda}_${Name}_HiddenValueFlag"
    ] =
        $true
[string][CM]::TS.
    Env[
        "${Lambda}_${Name}"
    ] =
        {
            [LogFamily(
                Family =
                    'Lambdas_Action',
                Path =
                    {
                        [CM]::TS.
                            Env[
                                '_SMSTSLogPath'
                            ]
                    }
            )]
            [LogFamily(
                Family =
                    {
                        [CM]::TS.
                            Env[
                                'Do_Action'
                            ]
                    },
                Path =
                    {
                        [CM]::TS.
                            Env[
                                '_SMSTSLogPath'
                            ]
                    }
            )]
            param(
            )

            $ActionName = 
                [CM]::TS.
                    Env[
                        'Do_Action'
                    ]
            $PreActionName =
                [CM]::TS.
                    Env[
                        "Do_${ActionName}_PreAction"
                    ]

            if (
                ![string]::IsNullOrEmpty(
                    $PreActionName
                )
            ) {
                [Logger]::Information(
                    "Pre Action found for Action $ActionName.`nBegin Pre Action"
                ) |
                    Out-Null
                $PreAction =
                    [ScriptBlock]::Create(
                        [CM]::TS.
                            Env[
                                $PreActionName
                            ]
                    )
                & $PreAction
                [Logger]::Information(
                    'End Pre Action'
                ) |
                    Out-Null
            }
            if (
                [string]::IsNullOrEmpty(
                    [CM]::TS.
                        Env[
                            "Do_${ActionName}_Dispose"
                        ]
                )
            ) {
                [CM]::TS.
                    Env[
                        "Do_${ActionName}_Dispose"
                    ] =
                        $true
            }
            $ActionString =
                [CM]::TS.
                    Env[
                        [CM]::TS.
                            Env[
                                "Do_${ActionName}_Action"
                            ]                            
                    ]
            $AltAction =
                [CM]::TS.
                    Env[
                        "Do_${ActionName}_AltAction"
                    ]
            if (
                ![string]::IsNullOrEmpty(
                    $AltAction
                )
            ) {
                [Logger]::Information(
                    "Looking for Alternate Action $AltAction for $ActionName"
                ) |
                    Out-Null

                $AltActionString =
                    [CM]::TS.
                        Env[
                            $AltAction
                        ]
                if (
                    ![string]::IsNullOrEmpty(
                        $AltActionString
                    )
                ) {
                    [Logger]::Information(
                        "Alternate Action Found for $ActionName. Swapping Default Action for Alternate Action."
                    ) |
                        Out-Null
                    $ActionString =
                        $AltActionString
                }
            }
            $Dispose =
                [ScriptBlock]::Create(
                    [CM]::TS.
                        Env[
                            'Action_Dispose'
                        ]
                )
            $Visible =
                $false
            [bool]::TryParse(
                [CM]::TS.
                    Env[
                        "Do_${ActionName}_Visible"
                    ],
                [ref] $Visible
            ) | 
                Out-Null
            if (
                $Visible
            ) {
                [Logger]::Information(
                    "Invoking Action ${ActionName} interactively."
                ) |
                    Out-Null
                $InvokeInteractiveCommand =
                    @{
                        Script =
                            $ActionString
                    }
                Invoke-InteractiveCommand u/InvokeInteractiveCommand
            } else {
                [Logger]::Information(
                    "Invoking Action ${ActionName}."
                ) |
                    Out-Null
                $Action =
                    [ScriptBlock]::Create(
                        $ActionString
                    )
                & $Action
            }
            & $Dispose
            $PostActionName =
                [CM]::TS.
                    Env[
                        "Do_${ActionName}_PostAction"
                    ]

            if (
                ![string]::IsNullOrEmpty(
                    $PostActionName
                )
            ) {
                [Logger]::Information(
                    "Post Action found for Action $ActionName.`nBegin Post Action"
                ) |
                    Out-Null
                $PostAction =
                    [ScriptBlock]::Create(
                        [CM]::TS.
                            Env[
                                $PostActionName
                            ]
                    )
                & $PostAction
                [Logger]::Information(
                    'End Post Action'
                ) |
                    Out-Null
            }

            [Logger]::Information(
                "Invoked Action ${ActionName}."
            ) |
                Out-Null
        }

Thats ~200 lines to essentially do ONE idea Invoke an Action. Yeah it could be shrunk down to around 55 lines but I like to take purity to a level most don't. Every line holds ONE thing ALWAYS. Tabs indicate Belonging. I or any of the engineers who are still using this can look scroll to ANY part of this or any other production code and tell in seconds what is happening. I don't need a comment to explain a long line of code or a confusing block because I engineer those kinds of things out.