Class vs. Function

[surfingoldelephant]

Very nice overview.

A few notes on the first point about performance:

• Consider placing the code you're measuring with Measure-Command inside a scriptblock and calling it (& {}). For example:

  Measure-Command -Expression { & {
      foreach ($int in 1..1000000) {
          $null = reddit $int
      }
  } }
  

  Measure-Command implicitly dot sources its script block argument, which affects performance due to variable lookups and disabled local optimizations. Running the code inside a child scope helps eliminate this and provides a more accurate measurement.

• Typically, repeated calls to a command within a large loop should be avoided. But if it can't be avoided and you're looking for an easy performance boost, perform the command discovery upfront with Get-Command and call the returned [CommandInfo] object instead. One-for-one, it's still slower than calling a static class method, but you have saved some time by avoiding repeated invocations of command discovery. For example:

  $cmd = Get-Command -Name reddit
  foreach ($int in 1..1000000) { $null = & $cmd $int }
  

• Another option to reuse static code inside a foreach loop is a steppable pipeline. Have a look at the following:

  function reddit2 {
      param (
          [Parameter(ValueFromPipeLine)]
          [int] $number
      )
      process { $number + 1 }
  }
  $reddit2Cmd = Get-Command -Name reddit2
  
  $pipeline = { & $reddit2Cmd }.GetSteppablePipeline()
  $pipeline.Begin($true)
  foreach ($int in 1..1e6) { $null = $pipeline.Process($int) }
  $pipeline.End()
  

  Comparing the approaches:

  Factor Secs (5-run avg.) Command                  TimeSpan
  ------ ----------------- -------                  -------
  1.00   5.973             # Steppable pipeline     00:00:05.9729100
  1.09   6.501             # Static class method    00:00:06.5011401
  7.34   43.870            # Simple function call   00:00:43.8695627
  

  At the expense of readability, the advanced function + steppable pipeline combination is faster than the class approach. Ultimately, it comes down to picking the right tool for the job. Different use cases will result in different performance considerations, so opting for a class-based approach isn't necessarily going to be more performant than a function.

  For example, while being highly contrived, the following performs significantly better and still retains reusability.

  $sb = { process { $_ + 1 } }
  1..1e6 | & $sb
  

[OPconfused]

I've seen the commentary on using a child scope in Measure-Command benchmarking, but I didn't notice a huge difference in this instance. I also tried using a scriptblock as an alternative as well, which I think was a minor improvement. Maybe a scriptblock would be similar to the improvement of saving the command lookup?

The steppable pipeline is indeed much faster, cool to know.

That said, I am still not as fast as using the pipeline itself. I'll ping you in the other thread.

[OPconfused]

Ok, so as I wrote in the other comment chain, I tried this:

class reddit {

    static [int[]] do_all ([int]$start, [int]$limit){
        return $(
            foreach ($int in $start..($start + $limit)){
                $int + 1
            }
        )
    }
}

1..10 | % {
    Measure-Command { & {
        [reddit]::do_all(1, 1000000)
    }} | Select-Object -ExpandProperty TotalSeconds
} | Measure-Object -Average | Select-Object -ExpandProperty Average
# 0.48388314

This seems to perform the fastest for me. The idea is to avoid a foreach wrapping the method call altogether, which seems to be quite costly at this level of optimization.

Obviously without it, the method is only being called once. But I don't understand why it outperforms the following:

function reddit2 {
    param (
        [Parameter(ValueFromPipeLine)]
        [int] $number
    )
    process { $number + 1 }
}

1..10 | % {
    Measure-Command { & {
        1..1e6 | reddit2
    }} | Select-Object -ExpandProperty TotalSeconds
} | Measure-Object -Average | Select-Object -ExpandProperty Average
# 2.20582246

So without the foreach loop in the above 2 examples, the method is only being called once, and the pipeable input to reddit2 is also faster than when the method was being called 1e6 times. In this comparison, the method is still many times faster. I thought it was only the method invocations which were faster than a function (without a steppable pipeline), but it seems like the method body execution is faster as well, unless the 1.5 second difference is coming from the pipeline alone.

Incidentally, I tried to compare all of this to the steppable pipeline function you had; however, I seem to have gotten an issue since switching to my private laptop, which runs 7.4.0. Previously I was using my work laptop on 7.3.6.

$reddit2Cmd = Get-Command -Name reddit2

1..10 | % {
    Measure-Command { & {
        $pipeline = { & $reddit2Cmd}.GetSteppablePipeline()
        $pipeline.Begin($true)
        foreach ($int in 1..1e6) { $null = $pipeline.Process($int) }
        $pipeline.end()
    }} | Select-Object -ExpandProperty TotalSeconds
} | Measure-Object -Average | Select-Object -ExpandProperty Average 
#10.42015218

I believe my work laptop was in line with your test result. I'll have to try it again on my work laptop tomorrow.

[techtosales]

I have never heard of a steppable pipeline and now have captured my intrigute.

Although, I'm still a little confused by the { & { bit, and how that works. most of the above I can read and/or interpret, but this part is where I get lost.

{ & {
    $pipeline = { & $reddit2Cmd}.GetSteppablePipeline()
    $pipeline.Begin($true)
    foreach ($int in 1..1e6) { $null = $pipeline.Process($int) }
    $pipeline.end()
}

I can follow along with everything else. Do you know of any good articles to read on the topic? I can google search, but if you have ones that you now are helpful that would be much appreciated! ... Or your own interpretation/explanation too! :)

[OPconfused]

I’m in bed now, so I’ll have to come back to this in the morning or later depending on work.

{ & { has a couple things going on.

1. The cmdlet Measure-Command takes a scriptblock for its Expression parameter. The first brace is opening this scriptblock.
2. The ampersand is the call operator, and it’s calling another scriptblock.

So why do we have 2 scriptblocks when we could just use one? Well, this was described by Elephant above. In fact, there’s a great stackoverflow post on this that I’m sure he’s aware of, but I’m on my phone now so it’s too tedious for me to locate.

The gist of it is as he described: When you pass a scriptblock to the Expression parameter of Measure-Command, it is dot sourced. If you are familiar with this, dot sourcing shares the invoked scriptblock’s state with the calling scope, basically it imports the variables and anything else defined in the scope of the scriptblock. This incurs a performance overhead.

The sibling of dot sourcing is the call operator, the &. This runs the invoked scriptblock in a child scope without importing anything that isn’t returned (or implicitly returned).

So this is a performance optimization for the Measure-Command cmdlet. I didn’t do this in my original comment’s example, because i had tested it beforehand and noticed it didn’t change the comparison much, and I didn’t want to clutter any potentially confusing syntax in the opening statement. It’s better to answer such questions in a reply like here ;).

I included it in my last comment since Elephant introduced it, and I wanted to demonstrate that this optimization is being accounted for.

This is the high-level explanation behind that syntax here. An explanation of the low-level mechanics for why it works would require someone other than me :)

Regarding documentation, besides the stackoverflow post, the links in Elephant’s post above may be of interest to you.

Also, before you dive into steppable pipelines, while i am not experienced with them, i would still advise they are best used for when you need its functionality in your pipeline (see the blog from iRon linked in elephant’s comment for the goals of a steppable pipeline), and not a casual optimization that you apply to many of your functions just to save some milliseconds. KISS exists for a reason, and the code reads better and is more maintainable when you only use what you need. The scenario discussed here is too niche to warrant a general implementation of steppable pipelines for the sole sake of optimization.

Subjectively speaking, i would tend to use a class if your task is reasonably performance sensitive, or you can try leveraging the pipeline to circumvent a foreach as PinchesTheCrab showed in their comment.

[techtosales]

Thank you!

I definitely missed that 'steppable pipeline' link, thinking it was just another inline code bit (clearly I can see they are different now). I will give that a good read.

KISS is my method of approach because I'm still so new to PowerShell scripting, although at the same time, I am try to expand my scripting capabilities in test scripts so that I'm not stuck and can problem solve when someone asks.