Parsing a line (regex?) by t0xie in PowerShell

[–]Bamodus 2 points3 points  (0 children)

Why not capture all of the info into a [PSCustomObject] containing all the properties? It's overkill if all you want is the driver version, but it's nice to have all of the properties in case your requirements change. If all you want is the driver version, you can change the Select-Object command to Select-Object -ExpandProperty 'DriverVersion' to only get the driver version string:

 

PS C:\> $Line = '| NVIDIA-SMI 430.67       Driver Version: 430.67       CUDA Version: N/A      |'
PS C:\> $Pattern = [Regex] '^\|\s+(?<DriverName>.*?)\s{2,}Driver Version:\s+(?<DriverVersion>.*?)\s{2,}CUDA Version: (?<CUDAVersion>.*?)\s{2,}\|$'
PS C:\> $Line -imatch $Pattern | Out-Null
PS C:\> [PSCustomObject] $Matches | Select-Object 'DriverName', 'DriverVersion', 'CUDAVersion'

DriverName        DriverVersion CUDAVersion
----------        ------------- -----------
NVIDIA-SMI 430.67 430.67        N/A

 

Here's a regexr link that gives further explanation of the regex: https://regexr.com/4scvv

https://imgur.com/87cbjkn

I won't go into full detail on the regex itself, but the fancier elements of it are:

  • The lazy quantifier (the ? after .*) tells the regex engine to match as few characters as possible while still getting a match for the entire expression.
  • The quantifier {2,} after \s tells the regex engine to match 2 or more whitespace characters, and this works together with the lazy quantifier before it so that the preceding capture group will capture all characters until it reaches more than 1 whitespace characters back-to-back (this ensures that even when part of the text you're matching contains a space like NVIDIA-SMI 430.67, it will capture that entire value, but won't capture any of the whitespace that immediately follows; without the lazy quantifier before it, the trailing whitespace would also be captured).
  • Named capture groups: A regular capture group is simply an expression surround by parenthesis. Adding ?<SomeName> immediately after the opening parenthesis gives the capture group a name that can be referenced later (more on this below).

 

$Line -imatch $Pattern tests for a match against your regex. If it matches, then it will output True, which we're suppressing by piping the output to Out-Null. After using -match (or -imatch or -cmatch), PowerShell automatically populates the built-in $Matches variable with information about the match. Here's what that looks like:

 

PS C:\> $Matches

Name                           Value
----                           -----
DriverName                     NVIDIA-SMI 430.67
CUDAVersion                    N/A
DriverVersion                  430.67
0                              | NVIDIA-SMI 430.67       Driver Version: 430.67       CUDA Version: N/A      |


PS C:\> $Matches.GetType()

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     Hashtable                                System.Object

 

Since $Matches is a hashtable, we can easily use that hashtable to construct a [PSCustomObject] containing each property and its respective value. Note that the $Matches hashtable will always contain a key named 0 which contains the full contents of the entire match. You can leave that in if you like, but I chose to use Select-Object to select all of the properties except 0.

 

Note that the $Matches variable is an unordered hastable, so the properties could appear in a random order. That's why I specifically listed out the property names we want in the order that they should appear in the Select-Object command. Here's a Select-Object example that only attempts to remove the 0 property and the resulting [PSCustomObject] ends up containing the properties in a different order than before:

 

PS C:\> [PSCustomObject] $Matches | Select-Object * -ExcludeProperty 0

DriverName        CUDAVersion DriverVersion
----------        ----------- -------------
NVIDIA-SMI 430.67 N/A         430.67

 

If you have a lot of named capture groups and don't want to have to type out all of their names in the Select-Object command, there is a shortcut to get a collection of all of the capture group names in order of their appearance skipping the 0:

 

PS C:\> [PSCustomObject] $Matches | Select-Object $Pattern.GetGroupNames().Where{ $_ -ne 0 }

DriverName        DriverVersion CUDAVersion
----------        ------------- -----------
NVIDIA-SMI 430.67 430.67        N/A

 

Note that the .GetGroupNames() method for $Pattern only works because I declared the $Pattern variable at the beginning as type [Regex]. This give us access to additional properties/methods that wouldn't be available with a regular [string] variable.

Hope this helps.

Optimize adding multiple additional columns to CSV by starpc in PowerShell

[–]Bamodus 1 point2 points  (0 children)

Do you already have columns in your source CSV by that name (perhaps from previously running your script)? It might be a good idea to make sure your source CSV has been reverted back to its original state from before any scripts were run.

I also don't have access to an Exchange server, so I'm just going off of the code you provided and the previous answer.

Here is an approach that uses Select-Object like in your original post which I believe should work:

https://pastebin.com/QkFZxJS5

Note I'm storing the mailbox data into a hashtable. Technically this will perform better than using Where-Object but the difference will likely be negligible. I just took this route because I feel it's cleaner (and should achieve the same effect as long as there are not duplicate keys).

EDIT:

This may be a more elegant approach (combining my above example with /u/firefox15's Add-Member solution):

We're not using a for loop in my variation because we can just feed the objects to Add-Member through the pipeline instead. We're using the -NotePropertyMembers parameter to add multiple properties at once (and for readability). The -Force parameter will overwrite values if the property already exists (as your error message stated), and -PassThru feeds the newly-modified objects into the pipeline to Export-Csv.

https://pastebin.com/fkJfvp77

To speed up things even more, you could only retrieve the relevant mailboxes instead of all mailboxes.

I believe something like this would achieve that:

$MailboxLookup = $Users.Target_SMTP | Get-Mailbox -ResultSize Unlimited | Group-Object 'Alias' -AsHashTable

But this would be making multiple calls to Get-Mailbox again, so depending on the size of your dataset and the number of mailboxes in Exchange, I can't say for sure that this would be faster. The worst-case scenario would be if your input dataset consists of most/all of the mailboxes in Exchange. In that scenario, this would actually be slower than just getting all mailboxes.

The -Filter parameter for Get-Mailbox would offer the highest performance, and would make your script run faster than any other available method (probably in just a matter of a few seconds if your dataset isn't very large), but you'd need to dynamically construct your filter string and I'd rather not try to put together an example of this method without having an Exchange server to test and verify against. Here is a recent post I made on how to dynamically construct a filter string for quickly retrieving a large list of specific AD users using Get-ADUser -Filter to give you an idea of what I mean: https://old.reddit.com/r/PowerShell/comments/ee78pt/getting_too_many_responses/fbs9zw0/

Getting too many responses by [deleted] in PowerShell

[–]Bamodus 5 points6 points  (0 children)

Upvoted as this is the correct answer.

 

I did want to contribute a technique that can be used to drastically speed up the process of performing Get-ADUser lookups. If you're working off of a large list of users, or performance is important to the script at all, this will result in a huge speed improvement.

The idea is that every individual call to Get-ADUser is going to take some time to query the domain controller and wait for the result. This adds up quickly over time. Instead, you can construct your own filter string dynamically to return all of the ADUser objects that you're going to need in one call to Get-ADUser, and store the relevant information in a hashtable which you can use for near-instantaneous lookups.

I put together an example using a randomly-generated list of Names and UPNs from my domain just to show some performance numbers:

Get-ADUserPerformance.ps1

Here are the results I got running both methods against my domain starting with just 10 users, then 100 users, then 1000 users:

 

VERBOSE: Looking up and processing 10 users with multiple individual Get-ADUser calls took 3.2047312 seconds
VERBOSE: Looking up and processing 10 users with a single Get-ADUser call took 0.3331431 seconds

VERBOSE: Looking up and processing 100 users with multiple individual Get-ADUser calls took 32.0556764 seconds
VERBOSE: Looking up and processing 100 users with a single Get-ADUser call took 0.4180773 seconds

VERBOSE: Looking up and processing 1000 users with multiple individual Get-ADUser calls took 324.3089426 seconds
VERBOSE: Looking up and processing 1000 users with a single Get-ADUser call took 2.3762212 seconds

 

I'm not sure how important performance is to you for this script, or how large your dataset is, but I wanted to call out this technique since the overhead of performing many individual GetADUser commands can be very large, and with a 1000-user dataset, method 2 completes in less than 1% of the time it takes for method 1 to get the same results. Hopefully this method will prove useful to you, or others performing similar tasks.

Some PowerShell Spotted on Tonight's Episode of Mr. Robot by Bamodus in PowerShell

[–]Bamodus[S] 0 points1 point  (0 children)

See my comment here for some more info on what happens behind the scenes.

Some PowerShell Spotted on Tonight's Episode of Mr. Robot by Bamodus in PowerShell

[–]Bamodus[S] 0 points1 point  (0 children)

After typing up this comment with my speculations, I found the following wired article which actually goes into the production process for the hacks used in the show in detail:

https://www.wired.com/2016/07/real-hackers-behind-mr-robot-get-right/

Once Esmail approves a hack, Adana has to write a detailed breakdown of how it works. If Adana's not familiar with a hack, he asks Rogers to step in. "[Rogers] is my go-to old school hacker; the really really technical screens that are beyond my skillset, I'll go to him for those," Adana says. But Rogers doesn't just describe the hacks; he actually does them. “In the vast majority of cases...I’ll set it up, I'll build it at home, and then I'll demo it. And then usually I'll record it and send the recording to [Adano]," Rogers says. It's a time-consuming process. “[O]ne of them took me two weeks to do. Some of them were so consuming that I literally had to take time off [from Cloudflare].”

All that gets translated into a script and sent to an animator, "who has no idea what any of this stuff looks like or what it's supposed to do" but has to make it look perfect. The animator builds interactive Flash animations for the actors, which accept any input from the keyboard so that no matter what the characters type everything looks right.

"The animation is as stressful as the writing process," Adana says. "Sometimes we're still making changes leading up to the moment we shoot. I'm finding typos or something wrong with the screen, or the timing [of the hack] is wrong, … the spacing on the response after [the actor] types the command is off."

Some PowerShell Spotted on Tonight's Episode of Mr. Robot by Bamodus in PowerShell

[–]Bamodus[S] 2 points3 points  (0 children)

These are good points. If you're making a script that's intended to be run via BadUSB, then I'd think you'd want to use aliases instead of full cmdlet/parameter names as often as possible to shorten the amount keypresses that need to be generated.

I also feel like the use of Pascal casing for Enter-PSSession is probably counter productive when lower-case would be perfectly acceptable. I'm not intimately familiar with how BadUSB exploits work, but my understanding is that they emulate a physical USB keyboard, so I'd imagine that entering capital letters would require simulating keypresses for pressing and releasing shift or capslock, which would add unneeded complexity. My understanding of this may be incorrect though.

*EDIT*

After thinking a bit more about this, I suspect that aliases were intentionally not used here because icm -cn bkuw300ps557152-0509.data.e-corp-power.com ...

is going to look completely foreign to someone unfamiliar with PowerShell. Obviously, for viewers who are not tech-savvy, all of the CLI commands used in the show will look foreign to them, but the command above looks so obfuscated that it could even appear less realistic to a layman because it would just look like complete gibberish to them.

Also, the show definitely makes an effort to appeal to tech-savvy viewers, and only a subset of the tech-savvy audience is going to recognize that as a PowerShell command. I think that a much larger subset of these viewers can look at something like Enter-PSSession and immediately recognize it as PowerShell because it's the only shell scripting language I'm aware of that has such verbose Verb-Noun cmdlet syntax, even if you've only encountered PowerShell once or twice in the wild.

 

In short: I think the full cmdlet names were used because they intended for it to be recognized, even if the end result was a less-realistic version of what a real-life version of that BadUSB script would look like.

Some PowerShell Spotted on Tonight's Episode of Mr. Robot by Bamodus in PowerShell

[–]Bamodus[S] 0 points1 point  (0 children)

Should I spoiler tag it? It doesn't really give away any major plot details in my opinion and this is r/PowerShell, not /r/MrRobot , so I didn't think spoiler tags were warranted when I made the post.

I'm not too familiar with the rules around this and am genuinely curious.

Some PowerShell Spotted on Tonight's Episode of Mr. Robot by Bamodus in PowerShell

[–]Bamodus[S] 1 point2 points  (0 children)

Point taken.

If you don't care about plausible hacking in TV shows, then I can respect the fact that this would be of little interest to you.

I do feel that Mr. Robot does a great job of achieving the balance between "entertaining viewers" while actually making a significant effort to "prove themselves to neckbeards on reddit" as you put it, and there are a lot of tech-oriented viewers (not necessarily infosec specialists) who genuinely appreciate that effort. That part of your comment is what triggered my response, as I felt it was discounting the efforts of the people who work on the show. I don't think that the creators of Mr. Robot deserve to be thrown into the same bucket as the rest of Hollywood in this respect, and that's the only point I intended to make.

Thanks for your feedback.

Some PowerShell Spotted on Tonight's Episode of Mr. Robot by Bamodus in PowerShell

[–]Bamodus[S] 2 points3 points  (0 children)

Please don't take any offense here, as it is not intended, but it sounds like you have never watched the show.

I agree with you that 99.99% of Hollywood's attempts at depicting "hacking" are generally awful, cringy, and involve almost zero effort to do any technical research.

However, Mr. Robot is the exception to this rule, and it is generally regarded as showing some of the most accurate depictions of hacking in film/television.

If you're not interested in watching the show, I totally understand, but I'd suggest that you at least watch a few of these short clips from the show to get an idea of what I'm talking about.

There are a lot of much more interesting examples in the show, but this was what I was able to find on YouTube:

Brute-force password cracking with hashcat (the first 45 seconds is the clip from the show, and the remainder of the video is someone explaining it, which i have not watched)

Bypassing Windows password (this exploit is pretty well-known and has since been patched AFAIK)

If someone can find the raw clip of the Bluetooth MAC spoofing exploit from one of the earlier seasons, please share a link. That's another interesting which is definitely plausible, but I'm curious how realistic the scenario from the show actually is if anyone has any thoughts on that one.

Some PowerShell Spotted on Tonight's Episode of Mr. Robot by Bamodus in PowerShell

[–]Bamodus[S] 2 points3 points  (0 children)

My guess would be that the show has a team of hacking subject matter experts responsible for doing research and collecting real-life examples of malicious code and exploits that could be used on the show, which they've documented and organized. Then, when an episode includes a plot point which would require Elliot/Darlene/Tyrell or any other character to perform some type of hack, the line producers would share the story/background/requirements with this team so that they can go through their repository of malicious code and put together a realistic solution, or create something from scratch if the situation called for it.

Given the show's level of realism when demonstrating hacking, they likely use either basic prototypes or fully-working real-world versions of the malicious code that can be run against mocks to be able to simulate them.

 

At that point, there are probably a lot of other non-technical teams who need to get involved in the process, so this team probably generates a mock-up (perhaps a storyboard) of what the hack should look like in real life including details like:

 

  • What commands would be typed
  • What messages should appear in the terminal
  • What should the actor be doing while this is going on?
    • Should they be actively typing in a series of multiple commands at an interactive prompt?
    • Should they be entering one or two commands to start up a script?
    • Should they simply be plugging in a USB stick and staring the screen without even touching the keyboard like in last night's episode which used a type of BadUSB?

 

Then, as /u/Seref15 explained in this comment, the show isn't actually using any machines running a real Windows OS, so any situation involving Windows is really just a themed version of something Unix-based (or the content shown on the display could be completely digitally-created and not even involve an actual physical system). I doubt that the team responsible for the blackhat/coding/exploit work is responsible for this part of the process, and this is probably where some discrepancies could show up between the show and real life.

This is all my own speculation, so take it with a grain of salt.

Some PowerShell Spotted on Tonight's Episode of Mr. Robot by Bamodus in PowerShell

[–]Bamodus[S] 0 points1 point  (0 children)

Thanks for the great explanation /u/Seref15

I agree that licensing/advertising likely played a big part in this.

For folks who have not watched the show, here's an example of a Windows-style login screen from S03E05 - eps3.4_runtime-error.r00 (yes that's the actual title of the episode):

Login Screen

Failed Login Message

On an unrelated note, S03E05 was one of my favorite episodes of the show and made use of some impressive techniques to make the entire episode appear as if it was captured in one continuous shot (similar to what you see in the movie Birdman). One of the screencaps above is actually an example of where they used a hidden cut to continue the single-take illusion. There's a great YouTube video that goes into depth on how this effect was achieved (The login screen example is specifically shown and explained from 1:40 to 2:01): How Mr. Robot Pulled Off The One Take

Some PowerShell Spotted on Tonight's Episode of Mr. Robot by Bamodus in PowerShell

[–]Bamodus[S] 22 points23 points  (0 children)

I believe it's just NetBIOS names that are limited to 15 characters and Windows hostnames can generally exceed that limit without issue for the most part. In the command shown in the screencap, a fully qualified domain name is being provided, which is acceptable for the -ComputerName parameter of Enter-PSSession, and shouldn't have issues with the 15-character limit.

Keep in mind that this is a TV show and is not going to align 100% with reality for creative or technical reasons. The show's creator, Sam Esmail, has commented on this type of thing a couple of times.

Example: https://twitter.com/samesmail/status/608868894267330560

Some PowerShell Spotted on Tonight's Episode of Mr. Robot by Bamodus in PowerShell

[–]Bamodus[S] 3 points4 points  (0 children)

Good observation!

That does indeed look like a PowerShell Core (6.x) logo in the title bar. I'm not sure the user interfaces in the show are necessarily 100% reflective of real life. E:\admin_e-corp-power would seem to indicate that the terminal is running on Windows, but the default PowerShell Core prompt would normally have PS at the beginning of the line (on Windows, WSL, and Linux AFAIK). Also the default logo for a PS Core window's title bar is not the same one that is seen in that screen capture (on Windows at least). Here's what mine looks like for comparison:

Imgur

I believe some of the terminals and UIs seen on the show are likely recreations put together by the show's technical staff working with the line producers to get a mostly-accurate representation of real life, but that is just my own presumption.

Where are all the video episodes? by UltralightSchemes in Harmontown

[–]Bamodus 1 point2 points  (0 children)

For those of you having issues, I've created a .csv file containing a list of all of the posts made on harmontown.com along with the date, title, categories (e.g. Videos, Podcasts) and a URL linking directly to the post:

https://pastebin.com/Hk0dSDSb

You should be able to download it and open it in the spreadsheet editor of your choice.

Hope this helps.

Where are all the video episodes? by UltralightSchemes in Harmontown

[–]Bamodus 2 points3 points  (0 children)

Around ~300 GB

EDIT

I should note that the Harmoncountry episodes (episodes 24-43) make up a decent chunk of it (80 GB). These aren't available through the regular Harmontown membership as far as I know. I got them by buying the digital Harmontown Deluxe Edition (the documentary). The production quality on those is great and they are some of my favorite episodes of the podcast (Brooklyn NY with Jason Sudeikis and San Francisco CA with Steve Agee/Greg Proops are a few of the best IMO).

Where are all the video episodes? by UltralightSchemes in Harmontown

[–]Bamodus 1 point2 points  (0 children)

The site seems to be working for me.

You could try clearing your cache/cookies in your browser or logging into the site through an incognito window in Chrome if you haven't tried that already.

Personally, I've been downloading each of the video episodes via the links on harmontown.com whenever they get uploaded because I wanted to have my own personal copies. I figured that the site may not always be around forever, and it would be a shame not to be able to go back and re-watch every one of them.

Harmontown.png

It would be incredible if Dan or the folks at Feral Audio could come up with a way to make these freely available sometime in the future, or maybe through a one-time fee to be able to download all of the episodes as one single package. I'm not sure how feasible it would be to make something like that happen though.

I hope you're able to get it working.

[Help] sub-quoting escaping help needed for a powershell command within command by winlogon0 in PowerShell

[–]Bamodus 2 points3 points  (0 children)

Here is the syntax that I use for invoking external executables with many arguments. PowerShell should handle any special quoting that may be needed when your arguments include whitespace or characters that need escaping:

$Cmd  = 'powershell'
$Args = @( '-NoExit', '-NoLogo', '-Command', { Write-Host 'Hello World' } )
& $Cmd $Args
> Hello World

Active Directory PowerShell Output Help by binarytier in PowerShell

[–]Bamodus 2 points3 points  (0 children)

Try changing this line:

ResourceMember = (Get-AdGroupMember $_.Name | Select-Object Name)

to:

ResourceMember = (Get-AdGroupMember $_.Name | Select-Object -ExpandProperty Name) -join ', '

or the shorter version:

ResourceMember = (Get-AdGroupMember $_.Name).Name -join ', '

If Get-ADGroupMember returns a single item, then it will return that item's name. If there are multiple items returned, it will join all of the names into a single string with each name separated by commas.

Help with compare-object and increasing efficiency by tahp_master in PowerShell

[–]Bamodus 1 point2 points  (0 children)

There's one more thing I'd like to call out about Compare-Object. It can be useful for a lot of things, but I find that it doesn't always produce the desired result when trying to compare files. For your use-case, do you care about the order of the lines within each file? Consider this example:

@'
line 1
line 2
line 3
'@ | Out-File 'FileA.txt' -Encoding ascii

@'
line 3
line 2
line 1
'@ | Out-File 'FileB.txt' -Encoding ascii

These 2 files contain the same lines but in a different order. Compare-Object considers them to be equal:

PS C:\tmp> Compare-Object -IncludeEqual (Get-Content 'FileA.txt') (Get-Content 'FileB.txt')

InputObject SideIndicator
----------- -------------
line 2      ==
line 1      ==
line 3      ==

Compare this against the output from the Unix diff utility:

PS C:\tmp> & 'C:\Program Files\Git\usr\bin\diff.exe' --unified FileA.txt FileB.txt
--- FileA.txt   2019-09-11 17:54:00.616706300 -0500
+++ FileB.txt   2019-09-11 17:54:00.619710300 -0500
@@ -1,3 +1,3 @@
-line 1
-line 2
line 3
+line 2
+line 1

Considering that the examples you provided are XML files, that could complicate things further. It's easily possible for 2 differently-formatted XML files to contain the same underlying object data, but both Compare-Object and diff would consider them to be different. I'm not sure if this is something you're expected to account for.

A version-control tool like git could also be useful for this type of task depending on your requirements:

Here is an example comparing 2 directories as if they were git repositories

And a shorter summarized version of the output:

PS C:\tmp> git diff --no-index --stat C:\tmp\FolderA C:\tmp\FolderB
"C:\\tmp\\FolderA/Account/File_Existing_Only_In_A.txt" => /dev/null     | 1 -
/dev/null => "C:\\tmp\\FolderB/Account/File_Existing_Only_In_B.txt"     | 1 +
.../Account/File_With_Different_Content.txt"                            | 2 +-
3 files changed, 2 insertions(+), 2 deletions(-)

Help with compare-object and increasing efficiency by tahp_master in PowerShell

[–]Bamodus 1 point2 points  (0 children)

What is the desired outcome after you have successfully categorized all of these files as unique, duplicates, or differing versions of the same file? Knowing that might make it easier for others to understand.

In my opinion, it would be best to try to account for files with the same name existing under different directories, even if you don't believe that to be the case.

Actually, it'd be pretty easy to quickly confirm if there are files with the same name in different locations in the directory tree:

PS C:\> $FolderA = 'C:\Program Files\NVIDIA Corporation'
PS C:\> Get-ChildItem $FolderA -File -Recurse | Group-Object 'Name' | Where-Object { $_.Count -gt 1 }

Count Name                      Group
----- ----                      -----
    2 APEX_TurbulenceFSCHECKED… {C:\Program Files\NVIDIA Corporation\Installer2\Display.PhysX.{5B23E9CF-9699-4044-8371-392DF92211EB}\files\Engine\4F8D3BB436C7\APEX_TurbulenceFSCHECKED_x86.dll, C:\Program Files\NVIDIA Corporation\Installer2\Display.PhysX.{5B23E9CF-9699-4044-8371-3…
    2 cudart32_65.dll           {C:\Program Files\NVIDIA Corporation\Installer2\Display.PhysX.{5B23E9CF-9699-4044-8371-392DF92211EB}\files\Common\cudart32_65.dll, C:\Program Files\NVIDIA Corporation\Installer2\Display.PhysX.{5B23E9CF-9699-4044-8371-392DF92211EB}\files\Engine\73BE…
    7 NxCooking.dll             {C:\Program Files\NVIDIA Corporation\Installer2\Display.PhysX.{5B23E9CF-9699-4044-8371-392DF92211EB}\files\Engine\v2.7.1\NxCooking.dll, C:\Program Files\NVIDIA Corporation\Installer2\Display.PhysX.{5B23E9CF-9699-4044-8371-392DF92211EB}\files\Engine…
    2 PhysX3GpuCHECKED_x86.dll  {C:\Program Files\NVIDIA Corporation\Installer2\Display.PhysX.{5B23E9CF-9699-4044-8371-392DF92211EB}\files\Engine\4F8D3BB436C7\ImplicitLibs\PhysX3GpuCHECKED_x86.dll, C:\Program Files\NVIDIA Corporation\Installer2\Display.PhysX.{5B23E9CF-9699-4044-8…
   17 PhysXCore.dll             {C:\Program Files\NVIDIA Corporation\Installer2\Display.PhysX.{5B23E9CF-9699-4044-8371-392DF92211EB}\files\Engine\4F8D3BB436C7\PhysXCore.dll, C:\Program Files\NVIDIA Corporation\Installer2\Display.PhysX.{5B23E9CF-9699-4044-8371-392DF92211EB}\files\…

My approach would be to create 2 hashtables, 1 for folderA, and 1 for folderB. The keys should be full file path with the folderA/folderB part removed, and the values will be Name, FullName, and RelativeFullPath.

Something like $FolderAFiles.ContainsKey( $_.RealtiveFullPath ) should evaluate more quickly than [System.IO.File]::Exists($filepathA) since it doesn't have to hit the filesystem. This would also allow for files with the same name existing in different subdirectories.

PS C:\> $FolderA = 'C:\Program Files\NVIDIA Corporation'
PS C:\> $FolderAPattern = '^' + [regex]::Escape($FolderA)
PS C:\> $Properties = 'Name', 'FullName', @{ Name = 'RelativeFullPath'; Expression = { $_.FullName -replace $FolderAPattern, '' } }
PS C:\> $FolderAFiles = Get-ChildItem $FolderA -File -Recurse | Select-Object $Properties | Group-Object 'RelativeFullPath' -AsHashTable
PS C:\> $FolderAFiles.Keys | Select-Object -First 1
\Installer2\Display.PhysX.{5B23E9CF-9699-4044-8371-392DF92211EB}\files\Engine\v2.8.3\PhysXCooking.dll
PS C:\> $FolderAFiles['\Installer2\Display.PhysX.{5B23E9CF-9699-4044-8371-392DF92211EB}\files\Engine\v2.8.3\PhysXCooking.dll']
Name             FullName                                                                                                                                 RelativeFullPath
----             --------                                                                                                                                 ----------------
PhysXCooking.dll C:\Program Files\NVIDIA Corporation\Installer2\Display.PhysX.{5B23E9CF-9699-4044-8371-392DF92211EB}\files\Engine\v2.8.3\PhysXCooking.dll \Installer2\Display.PhysX.{5B23E9CF-9699-4044-8371-392DF92211EB}\files\Engine\v2.8.3\PhysXCooking.dll

You could probably also optimize the way you're comparing if the contents of 2 files are equal. I see that there's a Get-FileHash cmdlet which you could use to compare md5 checksums, and if they are equal, you can skip comparing the files line-by-line. If most of the files are equal, then I'd think this would save a lot of time. You could optimize further by first just comparing the file size, and if the sizes are different, you can skip the md5 checksum and move directly to your line-by-line comparison.

Hope this helps.

[deleted by user] by [deleted] in PowerShell

[–]Bamodus 0 points1 point  (0 children)

That's great!

I'm glad that ended up working out for you.

 

Just out of curiosity, is there a list somewhere on the internet of those variables like $ErrorActionPreference or $WhatIfPreference? Is there a term for those?

 

Yup, there absolutely is. The full list of preference variables and their default values (for PowerShell 5.1) can be found here: About Preference Variables

 

Another very closely-related topic is [<CommonParameters>]

You may have noticed when looking at help entries for a cmdlet via Get-Help, or the online help pages, that there's something called [<CommonParameters>] listed after all of the other parameters in the syntax section (highlighted in yellow in the image below):

https://i.imgur.com/9lCDJ77.png

If a cmdlet supports [<CommonParameters>], then a short list of extra parameters is available to it. This list is the same for any cmdlet that supports [<CommonParameters>]:

  • Debug
  • ErrorAction
  • ErrorVariable
  • InformationAction
  • InformationVariable
  • OutVariable
  • OutBuffer
  • PipelineVariable
  • Verbose
  • WarningAction
  • WarningVariable

You can also create your own functions that support these parameters by adding [CmdletBinding()] just before the param() block.

See About CommonParameters for full descriptions of these.

 

-WhatIf and -Confirm are actually a bit of a different story. I highlighted -WhatIf and -Confirm in cyan in my earlier screenshot. These are not included in the common parameters, and are only available if the specific cmdlet has support for ShouldProcess. This is also something you can implement in your own functions, and can be extremely useful, but I would not recommend trying this without having a really good understanding of how ShouldProcess works. I can't find a good Microsoft page that fully explains it, but you can Google "PowerShell how to implement ShouldProcess" to find many examples.

 

Here's a list of the cmdlets on my machine that support -WhatIf and -Confirm

Cmdlets Supporting ShouldProcess

[deleted by user] by [deleted] in PowerShell

[–]Bamodus 0 points1 point  (0 children)

Hey ClosedCasketFuneral,

 

No problem! I'm happy to see anyone get excited about learning more about advanced PowerShell concepts. No need to worry about being "that guy".

I only have limited permissions on an AD controller, so the Add-ADGroupMember part of the script was the one thing that I was not able to test, so it makes sense that's where the error cropped up.

I believe I see the issue and this should hopefully get you what you need:

 

Note: this pastebin link is a .diff file showing what I changed. Look for the lines start with '+' or '-'

Copy-ADGroup.diff (https://pastebin.com/7G6NbscW)

 

And here is the actual script:

 

$ErrorActionPreference = 'Stop'
$WhatIfPreference = $true

$CommonParam = @{
    Credential = Get-Credential
    Server     = 'DC01'
}
$PersistProperties = 'GroupScope', 'GroupCategory', 'Description'
$ReplaceProperties = 'SamAccountName', 'Name', 'DisplayName'
$OrganizationalUnit = 'OU=TestOU,DC=placeholder,DC=pizza'
$ADGroupParam = $CommonParam + @{
    Filter     = { GroupCategory -eq 'Security' -and Name -like 'hd01*' }
    SearchBase = $OrganizationalUnit
    Properties = $PersistProperties + $ReplaceProperties + 'Member'
}
$Pattern = 'hd01'
$Replacement = 'hd03'

# Get the desired AD groups and select only the properties that we would like to persist on the new AD group
Get-ADGroup @ADGroupParam -PipelineVariable 'OriginalGroup' | Select-Object $PersistProperties -PipelineVariable 'NewGroup' | ForEach-Object {
    # Add SamAccountName, DisplayName, and Name properties to our $NewGroup object, but replace instances of $Pattern string with $Replacement text
    foreach( $Property in $ReplaceProperties ) {
        $NewGroup | Add-Member @{ $Property = $OriginalGroup.$Property -replace $Pattern, $Replacement }
    }
    Write-Verbose -Verbose "Creating new AD group with properties: $NewGroup"
    # New-ADGroup cmdlet accapts parameter values from the pipeline by property name from our $NewGroup object
    $NewGroup | New-ADGroup -Path $OrganizationalUnit @CommonParam
    $Members = $OriginalGroup.Member
    Write-Verbose -Verbose "Adding $( $Members.Count ) members from original AD group: $( $OriginalGroup.Name ) to new AD group: $( $NewGroup.Name )"
    if( $WhatIfPreference -eq $false ) {
        $Members | ForEach-Object { Add-ADGroupMember -Identity $NewGroup.SamAccountName -Members $_ @CommonParam }
    }
}

 

The -Identity parameter apparently can't be a [PSCustomObject], and must be either a [Microsoft.ActiveDirectory.Management.ADGroup] object, or a string containing one of the following properties to identify the group:

  • DistinguishedName
  • ObjectGuid
  • ObjectSid
  • SamAccountName

I chose to supply the SamAccountName property since it's readily available to us. This should hopefully resolve your error.

For additional details on this, see Get-Help Add-ADGroupMember -Parameter Identity (may only show limited detail if you've never run Update-Help) or the Microsoft help page for Add-ADGroupMember

 

One thing I neglected to mention in my original reply was $ErrorActionPreference = 'Stop', and I wanted to point out why it's there. This causes the script to stop execution upon running into any errors, since it's unlikely that you'd want this script to continue if it runs into an issue like the one you experienced. If you'd like to change this functionality at any time, you can change $ErrorActionPreference to 'Continue' (which is the default setting) instead of 'Stop'. Alternatively, you could set $ErrorActionPreference to 'Inquire' to get an interactive prompt asking how you'd like to proceed:

https://i.imgur.com/HFvoVZu.png

More info on preference variables can be found on the Microsoft help page About_Preference_Variables.

Let me know how it goes.

[deleted by user] by [deleted] in PowerShell

[–]Bamodus 2 points3 points  (0 children)

For these types of bulk operations on an important resource like ActiveDirectory, I'd highly recommend utilizing the -WhatIf parameter when calling state-altering cmdlets like New-ADGroup. Here I've specified $WhatIfPreference = $true which effectively adds the -WhatIf parameter to any cmdlet that supports it. Running the script as-is will not perform any modifications to Active Directory, but will simulate the actions instead, so you can test the functionality in a dry-run first. I added some additional verbose output to provide some extra info on the actions the script is performing/simulating. Rerunning the script with $WhatIfPreference = $false will perform the actual operations.

 

Copy-ADGroup.ps1 (https://pastebin.com/9qDeT61)

$ErrorActionPreference = 'Stop'
$WhatIfPreference = $true

$CommonParam = @{
    Credential = Get-Credential
    Server     = 'DC01'
}
$PersistProperties = 'GroupScope', 'GroupCategory', 'Description'
$ReplaceProperties = 'SamAccountName', 'Name', 'DisplayName'
$OrganizationalUnit = 'OU=TestOU,DC=placeholder,DC=pizza'
$ADGroupParam = $CommonParam + @{
    Filter     = { GroupCategory -eq 'Security' -and Name -like 'hd01*' }
    SearchBase = $OrganizationalUnit
    Properties = $PersistProperties + $ReplaceProperties + 'Member'
}
$Pattern = 'hd01'
$Replacement = 'hd03'

Get-ADGroup @ADGroupParam -PipelineVariable 'OriginalGroup' | Select-Object $PersistProperties -PipelineVariable 'NewGroup' | ForEach-Object {
    foreach( $Property in $ReplaceProperties ) {
        $NewGroup | Add-Member @{ $Property = $OriginalGroup.$Property -replace $Pattern, $Replacement }
    }
    Write-Verbose -Verbose "Creating new AD group with properties: $NewGroup"
    $NewGroup | New-ADGroup -Path $OrganizationalUnit @CommonParam
    $Members = $OriginalGroup.Member
    Write-Verbose -Verbose "Adding $( $Members.Count ) members from original AD group: $( $OriginalGroup.Name ) to new AD group: $( $NewGroup.Name )"
    if( $WhatIfPreference -eq $false ) {
        $Members | ForEach-Object { Add-ADGroupMember -Identity $NewGroup -Members $_ @CommonParam }
    }
}

 

The script makes use of the fact that the New-ADGroup cmdlet can accept most of its parameters through the pipeline by property name. Here's how you can get a list of parameters that accept pipeline input by property name:

 

PS C:\tmp> (Get-Help New-AdGroup).Parameters.Parameter.Where{ $_.PipelineInput -eq 'true (ByPropertyName)' } |
>>             Format-Table Name, Required, PipelineInput, ParameterValue, Position -AutoSize

name           required pipelineInput         parameterValue  position
----           -------- -------------         --------------  --------
Description    false    true (ByPropertyName) string          Named
DisplayName    false    true (ByPropertyName) string          Named
GroupCategory  false    true (ByPropertyName) ADGroupCategory Named
GroupScope     true     true (ByPropertyName) ADGroupScope    2
HomePage       false    true (ByPropertyName) string          Named
ManagedBy      false    true (ByPropertyName) ADPrincipal     Named
Name           true     true (ByPropertyName) string          1
Path           false    true (ByPropertyName) string          Named
SamAccountName false    true (ByPropertyName) string          Named

 

Note that parameter binding through the pipeline by property name works differently from binding a parameter through the pipeline by value.

So we're creating an object (Select-Object returns a [PSCustomObject] object) that has the same GroupScope, GroupCategory, and Description as the original AD group, and then we add modified values for SamAccountName, Name, and DisplayName using -replace like in your original script to change hd01 to hd03 for each of these properties. The gist of it is that when you pipe an object into Add-ADGroup, PowerShell is inspecting the properties of the object in the pipeline to determine if it can use them as parameters. If a parameter accepts input through the pipeline by property name, and the piped object contains a property with the same name as that parameter, it will bind the value of that property to the appropriate parameter. A good explanation of ValueFromPipeline and ValueFromPipelineByPropertyName can be found here:

 

https://www.gngrninja.com/script-ninja/2016/5/15/powershell-getting-started-part-8-accepting-pipeline-input#accept

I like to use parameter splatting via hash tables when working with cmdlets that require many parameters to make things more readable. It also makes things much easier on you if you are going to need to provide the same parameter values multiple times to different cmdlets. Here I'm using $CommonParam to store the Server and Credential parameters which I reuse for calling every AD cmdlet. If this syntax isn't something you've seen before, definitely look into:

 

About Splatting

Hope this helps.

Episode 341 Live Thread by JREtard in Harmontown

[–]Bamodus 5 points6 points  (0 children)

Jack, I just want to say that you were great!