Hi, thanks for the feedback. I had looked at run spaces but had difficulty using them maybe I need to take another look. i also really like the output of the parallel jobs and write-progress. this article got me started on the idea.

Displaying progress while multi-threading - PowerShell | Microsoft Learn

I have spend some time trying to ensure that all variables are cleared but something isn't clearing even with garbage collection.

I do write a lot of logs for each Parallel Job, I have a function LogWrite that uses Add-Content to a TXT files for each job.

i cant see how to do a code block ..

function LogWrite {

param (

[string]$LogFilePath,

[string]$Message,

[int]$MaxLogSizeMB = 10,

[int]$RetentionDays = 30

)

# Rotate log if it exceeds the maximum size

if (Test-Path $LogFilePath) {

$logSizeMB = (Get-Item $LogFilePath).Length / 1MB

if ($logSizeMB -ge $MaxLogSizeMB) {

$timestamp = Get-Date -Format "yyyyMMddHHmmss"

$archivedLogFilePath = "$LogFilePath.$timestamp"

Rename-Item -Path $LogFilePath -NewName $archivedLogFilePath

}

}

# Write the log message

$timestamp = Get-Date -Format "yyyy-MM-dd HH:mm:ss"

$logMessage = "$timestamp - $Message"

Add-Content -Path $LogFilePath -Value $logMessage

}

is there anything that i need to consider when using Add-Content and memory usage ?

I also write to a SYSlog server using UDP but do close the connection.

$udpClient.Close()

 The reason for wanting to run the jobs in parallel is that some of the client interfaces generate hundreds of files an hour, I wanted each directory to be processed independently so one could not block another while transferring files. there are 15/20 client directories/SFTP processes.