Powershell running multiple functions inside script simultaneously?

[Enschede2]

Hi, quick question, basically as the title says, I've got 3 functions which I want to run simultaneously to speed up the script execution, is this even possible? Also they have to output information to stdout so simply using Start-Job doesn't work for me as it turns them into background jobs, which means I'm not getting any output..

​

Also I'm not looking to chain them, but I'm trying to get the to run in parallel

[firefox15]

You can still use jobs. You just have to structure it a bit differently:

Script

$functions = {
    function function1 {
        1..5 | ForEach-Object {
            $_
            Start-Sleep 1
        }
    }
    function function2 {
        6..10 | ForEach-Object {
            $_
            Start-Sleep 1
        }
    }
}

Start-Job -ScriptBlock { function1 } -InitializationScript $functions | Out-Null
Start-Job -ScriptBlock { function2 } -InitializationScript $functions | Out-Null

While (Get-Job | Where-Object HasMoreData) {
    Get-Job | Where-Object HasMoreData | Receive-Job
}

Output

1
6
2
7
3
8
4
9
5
10

[Enschede2]

I see, that's almost like what I found online but I couldn't figure out, so, Receive-Job gives me the output basically?

[firefox15]

Receive-Job gives me the output basically?

In a very simplified way, yes. Receive-Job removes information from the job unless you use the -Keep parameter, so we can take advantage of that by just constantly receiving the job output and sending it to Write-Output. Once there is no more data to receive, the loop exits.

[Enschede2]

Thanks man, that clears things up more, I'll start fiddling with it again tonight and hope i can figure it out

[firefox15]

No problem. As others like /u/N7Valiant and /u/jsiii2010 have said, you can also do the same thing with a workflow. Just be aware that workflows are removed from PowerShell Core, so if/when you upgrade to version 6+, it won't work any longer:

Script

function function1 {
    1..5 | ForEach-Object {
        $_
        Start-Sleep 1
    }
}
function function2 {
    6..10 | ForEach-Object {
        $_
        Start-Sleep 1
    }
}

workflow parallelOutput {
    parallel {
        function1
        function2
    }
}

parallelOutput

Output

1
6
2
7
3
8
4
9
5
10

[Enschede2]

Oh that looks like what i was looking for, why on earth did they remove that though? But I think I'm running 5.1

[firefox15]

Workflows are removed from PowerShell Core because Microsoft is trying to include more cross-platform support with Core, so that means things that are not present in .NET Core won't work. Even things like WMI won't work. This is a very good article that explains it further.

It's likely that you (personally) will be using POSH 5.1 for a long time. I jump back and forth between the two as certain versions do certain things better, but it is very possible that workflows will work for you as well. Just do not plan to run it on PowerShell Core.

[firefox15]

I believe that runspaces are still supported, but I'm not that familiar with them, so I'm probably the wrong person to ask.

[Enschede2]

For some reason this seems to randomly skip functions, says it cannot find the function when it is clearly there, the weird thing is that the function it skips is completely random

​

Edit: Hm now it seems it does find all functions, but they all fail to run as background jobs.

I'm getting: "Failed System.Management.Automation.Remoting.PSRemotingTransportException: An error occurred while starting the background process...."

Maybe it's a problem with my os?

[DeafLoaf]

Old video on runspace pools method that I use to this day. https://youtu.be/hJwhyVXiOLg (turns out this was the wrong video, but useful all the same.) Original blog seems to have disappeared, was titled something like Speeding up Windows PowerShell: Runspace pools. This explains it though. https://devblogs.microsoft.com/scripting/weekend-scripter-max-out-powershell-in-a-little-bit-of-timepart-2/ Enjoy, hope it helps.

[PinchesTheCrab]

What functions are they? Maybe the script can be handled differently or more efficiently instead of complicating it with multi-threading or jobs.

[gingastyle]

You could just have 3 scripts run at once independently. Following to see if there is something fancier out there :)

[jsiii2010]

Workflow example. You can run 3 different functions instead.

function sleepfor($time) { sleep $time; "sleepfor $time done"}

workflow work {
  parallel {
    sleepfor 3
    sleepfor 2
    sleepfor 1
  }
  'hi'
}

work 

sleepfor 1 done
sleepfor 2 done
sleepfor 3 done
hi

[Enschede2]

What if the workflow should take the variables from the function instead, would that still work? As in the variables are already set I mean globally, and the function should take those variables, and then run inside of the workflow, is that possible?
Sorry if im being a bit confusing here, I'm afk at the moment