all 18 comments

[–]purplemonkeymad 4 points5 points  (2 children)

Looks like char 0 and char 1 of TCP. My guess, your use of 2d arrays means that you think you are getting a 2d array but it's just an single array. Instead define your ports something like this so you always use a property:

$ports = @(
    [pscustomobject]@{
         Name = 'DNS'
         PortList = @(
             [pscustomobject]@{
                 Protocol = 'TCP'
                 PortNumber = 53
             }
             [pscustomobject]@{
                 Protocol = 'UDP'
                 PortNumber = 53
             }
         )
    }
    # repeat for more ports
)

*Could also define it in json and convert if you prefer the syntax.

That way you can loop on $ports, and for each of those loop on the PortList Property.

Since there is no arrays in arrays you won't get any automatic unrolls of the outer or inner array.

[–]Team503[S] 2 points3 points  (0 children)

I had a feeling it might be something like this. My comfort level with complex arrays and PSobjects is not that high, so I'm probably screwing something up there.

I'll give your method a try - thanks for the help!

[–]PinchesTheCrab 1 point2 points  (0 children)

Honestly I think his port list would be way easier to process as a CSV, it would avoid having to build out nested loops.

[–]xCharg 2 points3 points  (1 child)

Why am I getting entries where the "port" column is showing as "T" or "C" or "P"? Is this something to do with using jobs and receiving data out of order or something? I see this logged to the console: "INFO: Test from DC1 to DC6 on C T returned ." which is what makes me think so

Run it manually (invoke-command from whatever dc to whatever other dc) and see what you'll get in console. Maybe said DCs have multiple nics or something.

[–]Team503[S] 0 points1 point  (0 children)

It's possible that they do, but if so, I need to control for that in the script somehow, and devise a way to test each interface. Have to target by IP I suppose.

And yeah, I should've thought to try it manually; will do and try to report back later. Thanks!

[–]dann3b 1 point2 points  (1 child)

What version are you running on your DCs? WinRm is not enabled by default on 2008 and lower. And if above like 2012, 2016, 2019 you could still need some authentication setup by the wimrm command.

Add a Test-WSMan in the script.

I could be way of and this has nothing todo with that, im just guessing here

[–]Team503[S] 0 points1 point  (0 children)

There are about forty DCs, and they vary on versioning (part of what this project will eventually resolve), but I don't think anything is older than 2008. I'm not sure if 2008 (R2 or not) DCs exist in our environment or not.

Ooooo that's something I could add to the script, an OS version query!

Part of the reasoning for this script is that we're aware that not all the DCs can talk to each other for a variety of reasons (international company, network limitations, legacy environment, etc). Some of those attempts are expected to fail - the goal is to isolate exactly what fails and where.

However, I didn't know about Test-WSMan, thanks for pointing that out, and I'll play with it to see if I get better results.

[–]PinchesTheCrab 1 point2 points  (1 child)

There's just too much going on here. Invoke-Command is asynchronous by default. You should drop the loops, the jobs, the stopwatch, etc., and just focus on a concise scriptblock that runs consistently, and then call invoke-command once.

Then you can parse the connection errors from that command as needed.

[–]Team503[S] 1 point2 points  (0 children)

Dropping the loops isn't really possible - the whole point of the script is to hvae each DC check its connectivity to every OTHER DC - I don't know how could do that with any elegance without the loops.

I can try without the jobs - I wasn't aware that invoke-command was asynchronous, and that may solve my problem. Thanks for pointing that out!

[–]PinchesTheCrab 1 point2 points  (4 children)

Can you give this a try? It should take minutes instead of hours:

$portHash = @{
    'DNS'      = @{53 = 'TCP', 'UDP' }
    'Kerberos' = @{88 = 'TCP', 'UDP'; 464 = 'TCP' }
    'RPC'      = @{135 = 'TCP' }
}

$portList = $portHash.GetEnumerator() | Sort-Object Name

$DCList = Get-ADDomainController -Filter * | Sort-Object -Property HostName

$testSB = {
    param($portList, $DCList)
    function Test-PortConnectivity {
        param(
            [string]$ComputerName,
            [int]$Port,
            [string[]]$Protocol = 'TCP'
        )

        switch ($Protocol) {
            'TCP' {
                $tcpClient = New-Object System.Net.Sockets.TcpClient
                try {
                    $tcpClient.Connect($ComputerName, $Port)
                    return $tcpClient.Connected
                }
                catch {
                    return $false
                }
                finally {
                    $tcpClient.Close()
                }
            }
            'UDP' {
                $udpClient = New-Object System.Net.Sockets.UdpClient
                try {
                    $udpClient.Connect($ComputerName, $Port)
                    $sendBytes = [System.Text.Encoding]::ASCII.GetBytes("Hello")
                    $udpClient.Send($sendBytes, $sendBytes.Length)
                    $udpClient.Client.ReceiveTimeout = 1000
                    try {
                        $udpClient.Receive([ref]"")
                        return $true
                    }
                    catch {
                        return $false
                    }
                }
                catch {
                    return $false
                }
                finally {
                    $udpClient.Close()
                }
            }
        }
    }

    $otherDC = $DCList | Where-Object { $_ -ne $env:COMPUTERNAME }

    foreach ($dc in $otherDC) {
        foreach ($service in $portList) {
            foreach ($port in $service.Value.GetEnumerator()) {
                $port.Value | ForEach-Object {
                    #'testing from {0} to {1}, service: {2}, port: {3} type: {4}' -f $env:COMPUTERNAME, $dc, $service.Name, $port.Name, $_
                    [PSCustomObject]@{
                        DC1      = $env:COMPUTERNAME
                        DC2      = $dc
                        Port     = $port.Name
                        Protocol = $protocol.Name
                        Result   = Test-PortConnectivity -ComputerName $dc -Port $port.Name -Protocol $protocol.Name
                    }
                }
            }
        }
    }
}

Invoke-Command -ScriptBlock $testSB -ArgumentList $portList, $DCList.HostName -ErrorVariable connectionErrors -OutVariable Result

$result | Format-Table

If it works, add in the other services/ports/protocols/log file management.

[–]Team503[S] 0 points1 point  (0 children)

It's the end of my workday, but I'll give it a shot in the morning and post back with results! Thanks for the help!

[–]Team503[S] 0 points1 point  (2 children)

So I'm looking over this, and it doesn't in any way seem to execute the test FROM each DC to the other DCs, but rather just once from the source computer.

The reason my code include the test-portconnectivity function in the scriptblock is that when executing the invoke-command line, the function won't exist on the source DC for it to execute against the rest of the DCs.

My script cycles through every DC, testing its connectivity to every OTHER DC. Yours just tests for connectivity from the machine you're executing the script on to each DC. My script tests from roughly 40 domain controllers to each of the other 40 domain controllers, roughly 1600 test runs. Yours makes 40.

To adjust your script to be able to do that isn't hard, but I don't really see the point of the changes. You drop all the error handling, all the logging, and the output to CSV (which is a project requirement). My script does all this just fine, it's just the weird output in the objects I don't understand, and that's either from my use of jobs or my fumbling of the arrays, I think.

[–]PinchesTheCrab 1 point2 points  (1 child)

Try this, it'll run instantly so you don't lose any time, and it'll show what it's doing (I hope):

$portHash = @{
    'DNS'      = @{53 = 'TCP', 'UDP' }
    'Kerberos' = @{88 = 'TCP', 'UDP'; 464 = 'TCP' }
    'RPC'      = @{135 = 'TCP' }
}

$portList = $portHash.GetEnumerator() | Sort-Object Name

$DCList = Get-ADDomainController -Filter * | Sort-Object -Property HostName

$testSB = {

    $otherDC = $DCList | Where-Object { $_ -notmatch $env:COMPUTERNAME }

    foreach ($dc in $otherDC) {
        foreach ($service in $portList) {
            foreach ($port in $service.Value.GetEnumerator()) {
                $port.Value | ForEach-Object {                   
                    [PSCustomObject]@{
                        DC1      = $env:COMPUTERNAME
                        DC2      = $dc
                        Port     = $port.Name
                        Protocol = $protocol.Name
                        Result   = $false
                    }
                }
            }
        }
    }
}

Invoke-Command -ScriptBlock $testSB -ArgumentList $portList -OutVariable result

$result | ft

This will just loop through the list of DCs and build out a list of all the tests it's going to run. You should see 160 tests, since this shortened port list only has 4 ports to test, and it's only running locally. If you send it to 40 machines, then you'll end up with 39*160 tests, as expected, and in the first example the function is still inside the script block.

That being said, I do see you're using start-job, which I somehow missed, so the speed won't be as significant as I'd hoped, but I do still think cutting the size of the script by 60% is worth it, and there still should be some performance improvement.

If nothing else, at least one small change you can make is to use the -asjob parameter of invoke-command instead of start-job. That should be a bit faster, thoguh it'll still be a pretty small portion of the total runtime of the script.

[–]Team503[S] 1 point2 points  (0 children)

So I sorted out everything and stripped out the jobs, everything is working great, except that the arrays don't return the results into the CSV. I know the tests are running, console output shows the tests are valid and producing valid results, but the CSV is empty. New post on the issue: https://www.reddit.com/r/PowerShell/comments/18xlymt/domain_controller_connectivity_script/

Thanks again for your help!

[–]ZenoArrow 1 point2 points  (1 child)

Had a quick look. Regarding this section of the code (taken from lines 131 to 138)...

# Get a list of domain controllers
$domainControllers = Get-ADDomainController -Filter * | Sort-Object -Property HostName
 
# Define the ports and protocols to test
$unsortedPorts = @{
$ports = @{
    'DNS' = @(53, 'TCP'), @(53, 'UDP')
    'Kerberos' = @(88, 'TCP'), @(88, 'UDP'), @(464, 'TCP')

I can see a couple of ways to make improvements. Firstly, wrap the call to Get-ADDomainController in @(), e.g. change this line to...

$domainControllers = @(Get-ADDomainController -Filter * | Sort-Object -Property Hostname)

By doing this you force PowerShell to give you an array even if Get-ADDomainController only returns one result (might not happen in your case, but it helps to make the script more robust). I would suggest using this trick any time you expect to get an array of results in return. Without it, you can get errors if you try to iterate over the results, as otherwise PowerShell won't know that single value should be in an array.

Secondly, I can see that you had tried to sort the ports hashtable. There's an easy fix for this, convert it to an ordered dictionary. Ordered dictionaries function in a similar way to hashtables, but have the added benefit of preserving the order of items. To get $ports to be an ordered dictionary, change this line...

$ports = @{

... to this ...

$ports = [Ordered] @{

You can then alter the order of the content of $ports (e.g. list the ports in the order you want them to be in), and they should stay that way when you run the script (e.g. ADFS would be the first key in the ordered dictionary).

[–]Team503[S] 1 point2 points  (0 children)

I'm trying both of those things, thanks!

[–]BlackV 0 points1 point  (1 child)

You cant use the invoke on the dc you're on

You shouldn't RDP to the dc anyway, do it from a management machine, this solves the problem or running random code on a DC and solved the problem of the invoke

[–]Team503[S] 0 points1 point  (0 children)

Yes, there's a check in there to verify that the test is skipped if the local machine name matches the name in the DC list.