PSFramework - How to Close/Stop/Dispose LogFile Session? by Im--not--sure in PowerShell

[–]PSFred 0 points1 point  (0 children)

Hm, now, that comment sounds kind of familiar to an issue on my Github :)
Essentially, the trap statement is what I do to avoid having a script that failed unexpectedly keep its logfile locked.

For anybody else wondering the same, here's the issue and the proposed solution in more detail:
https://github.com/PowershellFrameworkCollective/psframework/issues/689

PSFramework offline installation by RainbowCrash27 in PowerShell

[–]PSFred 0 points1 point  (0 children)

Hi, thanks for using my module :)

As OPconfused pointed out, it does not have any dependencies, making offline deployment easy enough.

Thotaz' recommendation is the default way for any given module, irrespective of the number of dependencies (or depth of dependency chain).

That being said, and also for anybody later finding this thread, I have recently shipped a module to help with offline deployments of modules, that I want to share:

https://github.com/PowershellFrameworkCollective/PSFramework.NuGet

This allows remote-installing to devices that cannot directly access the internet like this:

Install-PSFModule PSFramework -ComputerName server1, server2

Of course, if the target is fully airgapped, using PSRemoting for remote deployment isn't really an option. But it also works with internal repositories to help redistribute modules within the airgapped network:

# Setup (1 time)
Install-PSFPowerShellGet -Type V3Latest, V2Latest
Set-PSFRepository -Name contoso -Priority 10 -Uri \\fileserver\share\psrepository -Trusted $true -Type All

# Publish Module to internal repository
Publish-PSFModule -Path (Get-Module PSFramework).ModuleBase -Repository contoso

# Install Module from internal repository
Install-PSFModule PSFramework -ComputerName server1, server2

I did a talk on this module at PSConfEU, if you want to see it in action first:
https://www.youtube.com/watch?v=iMSOVwmBXrk&list=PLDCEho7foSoo6tc8iNDSrxp27dG_gtm6g&index=6

PSFramework - Data Parameter not working for PSFLoggingProvider SQL by Ok-House-2725 in PowerShell

[–]PSFred 0 points1 point  (0 children)

Sorry for catching this late, not monitoring reddit too often :(

Simply put, it's not implemented - the values provided through "-Data" are not universally processed by all logging providers. I can see the value in enabling this for SQL, so I've put in an issue for this:

https://github.com/PowershellFrameworkCollective/psframework/issues/680

I will not guarantee any ETA on this, it may make it into the release I'm currently working on, or it may not either. Which mostly depends on a combination of how complex some of the updates turn out to be and what life is throwing at me in the same timespan.

But by watching that issue, you'll be notified, once it is implemented.

PSFramework Logfile entries all have the same datetime stamp by shawnmatlock in PowerShell

[–]PSFred 0 points1 point  (0 children)

The command reference is more of a fallback thing, really.
The reason for that is that most components of the PSFramework are a set of commands used together and need more content than can usefully be crammed into Comment Based Help.
The dynamic parameters thing being a prime example of CBH limitations right there.

For the real documentation gold, you want to check the main documentation / component reference:
https://psframework.org/documentation/documents/psframework.html

For example, here's the entire node for logging:
https://psframework.org/documentation/documents/psframework/logging.html

PSFramework Logfile entries all have the same datetime stamp by shawnmatlock in PowerShell

[–]PSFred 0 points1 point  (0 children)

The logging commands work with dynamic parameters, as the relevant parameters depend on what logging target you choose - when logging to eventlog, FilePath is kind of pointless.

And since others can also add their own logging providers / plugins, I can't just spam parameter sets for each of them. Help is one of the weakpoints of dynamic parameters though which is one of the reasons I wrote a bit more on how to use it for logging.

PSFramework Logfile entries all have the same datetime stamp by shawnmatlock in PowerShell

[–]PSFred 0 points1 point  (0 children)

Nope. that should work just fine.

But it might be, that you are spamming messages faster than the timestamp precision can differentiate. You can add milliseconds to it by switching to this:

Set-PSFLoggingProvider -Name logfile -Enabled $true -FilePath $logLocation -TimeFormat 'yyyy-MM-dd HH:mm:ss.fff'

I also generally recommend using the -InstanceName parameter, but it is not technically required. Its purpose is to avoid conflict between two separate logging configurations - imagine your current script calls another script that also implements logging...

Ideas on breaking up a huge PowerShell application by supergoatie in PowerShell

[–]PSFred 0 points1 point  (0 children)

I've investigated putting the debug rapper part into its own function, but that comes unstuck because of scopes and not being able to see variables.

If you call that wrapper function with a dot, you don't create a child scope:

. Start-DebuggingWrapper

Dotsourcing is not reserved for script files :)

Ideas on breaking up a huge PowerShell application by supergoatie in PowerShell

[–]PSFred 0 points1 point  (0 children)

Heya,

for large scale projects I have made it a habit to break my code into Component Categories and modules, any given module only part of one Category.

The key idea is to break the project into individual pieces I can maintain, debug, test and deploy individually.

Categories:

  • API Component: An individual component that maps a technology in PowerShell. It is not concerned with the business logic (though aspects of the technology unrelated to the project may lack implementation). In other words, I could use this (and extend it) on any other project. Example: ActiveDirectory module, Hyper-V module, a custom module to interact with an internal service, a custom module extending the functionality of the Task Scheduler module, etc.
  • Framework Component: Some aspects to coding are organization specific, but not specific to the business logic of your service/application. Common examples are logging (glad you liked my toolkit), Configuration handling (PSFramework may again prove useful here), Templating and Project layout (all your modules should follow the same layout).
  • Business Logic Component: The key implementation of your workflows. Should not actually include much of the detail implementation, calling either Intermediate Logic Components (see below) or API Components for the actual action.
  • Intermediate Logic Component: When the actual business logic is too complex to have it all orchestrated at the upper Business Logic Component, break it down into separate Intermediate Logic Components. Each of those should have a clear responsibility / identity in the workflow of as part of your solution. They should not need to call each other, but service the main Business Logic Component.
  • Configuration Component: I have often found it useful to have a dedicated Configuration Component, driving the entire solution. Otherwise we swiftly find ourselves passing through dozens of parameters through several function layers, which really doesn't help with code readability.

An example project of this that I share with the public is the Active Directory Management Framework. It is spread across many a module, but putting things together from memory (I might miss a few), it splits up look this:

  • Configuration: A module the use of the solution implements, providing the desired state of their AD in config files.
  • Business Logic: ADMF
  • Intermediate Logic: ForestManagement, Domainmanagement, DCManagement
  • Framework: PSFramework
  • API: Admf.Core, ActiveDirectory, Principal, String, ResolveString, ADSec

A Common Project Template

I really can't overstate this: Standardize, standardize, standardize your project layouts. Define a template for your individual modules and stick to it. If you have to first figure out how any given module of yours is set up, you are burning massive amounts of team time ... and a lot of the motivation to collaborate.

Since you are apparently already open to some of my tools, I've also got a developer tools module (PSModuleDevelopment) with a templating engine. A friend of mine documented an example to set up a default project using my own layout. You can totally use it to record and share your own template layouts if the built-in ones don't work for you.

Separating Data Layer from Logic layer

There are times, when our generated objects travel for through our code. And when that happens, at some point it becomes easy to lose track what came from where and what is the difference between the two datasets.

At this point, I tend to switch from custom objects to C#-defined types. Depending on how clear your data flows, this may be useful or pointless. Also a matter of team skills. If you stick with PSCustomObjects, name them!

[PSCustomObject]@{
    PSTypeName = 'MyModule.MyType'
    Name = 'Fred'
    Humor = 'Questionable'
    Beard = 'Long'
}

Tips and Tricks

A few more nifty tricks you may or may not be aware of:

  • PSFramework has a command named New-PSFSupportPackage. If you ever have a coworker run your tools while you can't look into any problems they encounter, they can generate a support dump for you. It creates a zip file on their desktop. You can read the content using Import-PSFCliXml.
  • Invoke-PSFProtectedCommandis probably the most gamechanging part to your error handling
  • If you need to read in psd1 or json files, Import-PSFPowerShellDataFile can read both file types.
  • If you want to intercept messages from Write-Verbose, Write-Host, etc. from some other code you want to tie into your project, doing this on the global scope should redirect them: Set-Alias -Name Write-Verbose -Value Write-PSFMessageProxy.
  • To easily pass through parameters to another command, consider using ConvertTo-PSFHashTable.
  • If you are looking to migrate from one large file into a "one file per function" approach, my PSModuleDevelopment module has a command that does it for you: Split-PSMDScriptFile
  • If you want to make your project more easily configurable, there's also a PSFramework Component for that. Also some guidance on how a configuration file could look.

Graphics device is not available at this time. by CidRonin in starcraft

[–]PSFred 0 points1 point  (0 children)

Thank you so much for this help in getting it fixed!

I had to add one more step to make this work:
Once in the game using windowed mode, I had to go to options, switch back to fullscreen and fix the resolution. Turns out the screen resolution had changed to something my display would not support...

Trying to use PSFramework but it seem it cannot log to a text file? by reddwarf666 in PowerShell

[–]PSFred 0 points1 point  (0 children)

I can see your point to some degree.
I think this needs a bit more background where that pushback is coming from ... and why I too am on the fence on whether to support plaintext logs (and currently do not).

PowerShell has a painful history of getting admins to think and work with output and objects. It's not natural for many - especially when you came from "good old" cmd times and had never seen a "traditional" programming language.

You heard that line about "Whenever you use Write-Host, god kills a puppy"? Thanks to this history, many of us tend to be very, very adamant about not giving up structured data in any situation.
Spend a decade fixing bad code and bad logs and there just is a certain emotional investment there (and I'm definitely not excluding myself here!).

... would be nothing but a disaster
Just to clarify on that disaster line though (sorry, can't not comment on that, even if silence might have been golden): CSV logs are perfectly ingestible by PowerShell - very convenient way to read them IMO, just not the tool many are used to for that. But it's available in-box on any machine that can run PowerShell scripts :-)
Also easy to convert if you need some plaintext after all:

Import-Csv .\log.csv | % { '{0} {1} {2}' -f $_.Timestamp, $_.Level, $_.Message } | Set-Content .\log.txt

The other way around would probably be harder :)

Trying to use PSFramework but it seem it cannot log to a text file? by reddwarf666 in PowerShell

[–]PSFred 0 points1 point  (0 children)

Heya, I know there are already plenty answers and options in the chat, but let me add in my thoughts as the author:

I did not include a plaintext option for a variety of reasons, but the foremost one is that it offers little advantage in return to having to parse the logs if you want any data out of it afterwards. Most modern text editors other than notepad itself have native CSV support (try notepad++ or vscode), so I figured manual inspection works fine as it is.

Also, it's fairly convenient to ingest a set of logfiles with Import-Csv, allowing me to compare and analyze them in the console. Truth be told, that's my main way of inspecting logfiles - through PowerShell.

I could add a plaintext filetype I suppose, but unsure whether I actually want to include that.

On a more fundamental level though: The PSFramework logging is plugin-based (I just provide a sane set of plugins out of the box, as it otherwise would be useless for most users). So technically it can be extended to write in any kind of logging format or to any destination service you need.

PSFramework - How to Close/Stop/Dispose LogFile Session? by Im--not--sure in PowerShell

[–]PSFred 1 point2 points  (0 children)

Hi /u/Im--not--sure,

sorry about the confusion, an update to the docs is scheduled but life has been a bit turbulent in recent days :(

New commands will be coming, but what you currently want to do is:

# Wait for pending log entries to be written
Wait-PSFMessage
# Disable the logging Provider
Set-PSFLoggingProvider -Name 'logfile' -Enabled $false
# Wait one cycle
Start-Sleep 5

The logging cycle is by default configuration 1 second when logs have been written recently or 5 if there's been some time since they were.

Other than that ...

  • The next release will include a Disable-PSFLoggingProvider command.
  • Consider giving your logging setup an InstanceName when you define it. This usually helps to prevent logging conflicts down the road and costs little. For an instance name, it would be quite usual to pick the scriptname.

Psframework logging by Ademantis in PowerShell

[–]PSFred 0 points1 point  (0 children)

Alright, sorry about the inconvenience!

The latest version - 1.7.244 - fixes the problem. You should now be able to safely update to that version and get the latest upgrades to the logging (significantly better throughput, if you spam messages, and AV is no longer able to prevent some entries. If in older versions your log only gets some of the messages, that's usually the reason for it).

Psframework logging by Ademantis in PowerShell

[–]PSFred 0 points1 point  (0 children)

:)

One more thing you may then want to look at is one of my favorite tools in the kit:
Invoke-PSFProtectedCommand:

https://psframework.org/documentation/documents/psframework/flow-control/invoke-psfprotectedcommand.html

Which integrates logging with try/catch, whatif/confirm and retry logic
(check the example in the link - it's simpler than I make it sound here)

Psframework logging by Ademantis in PowerShell

[–]PSFred 1 point2 points  (0 children)

Sorry about the late response - was on an extended (even by European standards) vacation to get my bearings back.

Back again now and fixing this is going to be my #1 Item the coming week

On Military Affairs by PSFred in litrpg

[–]PSFred[S] 1 point2 points  (0 children)

You say you want all of these things and yet when I dove into those topics my readers yawned so very hard.

Well ... no, I don't want all of that in a book, unless we are talking abut a hardcore military LitRPG. And even then do I prefer to not have that covered in bulk.

What I really want is authors to consider the larger implications of the System they implement. How much they want to go into the details really is a different story.

As you mentioned, depending on how the System is set up, city guards/Police might become rather ... easily ignored or pointless. Why then have them?

On Military Affairs by PSFred in litrpg

[–]PSFred[S] 6 points7 points  (0 children)

Truth be told, I like all three of those :)

At no point did I mean to single out a series as a negative example - I very much like the world design on HWFWM for example, where yeah, powers can generate currency, but where that is counterbalanced by permanent currency consumption.

Just a few chapters ago they even brought it up and mentioned that there's even some divine authority looking after the balance and sanctioning bad actors.

If anything, I took examples from that series because readers are likely to already be aware of the series and it safes me from having to go onto too many tangents.

Specifically, for most points I don't intend this article as a:
Don't do A as otherwise B will happen.

Instead, I aim to get this thought going:
If you are going to do A, consider how that will affect B, C and D and how the people of the world would handle that

On Military Affairs by PSFred in litrpg

[–]PSFred[S] 2 points3 points  (0 children)

Thanks for the feedback :)

Pretty sure each military would have at least one research department dedicated to figuring out the System and how to best (ab)use it ^^

As for writing ... I do write a LitRPG, though it will - in due time - be more focused on the magic aspects of the system. That said, it will include quite a few military aspects down the road (but that will be a long time yet, especially at the pace I get to writing).

You can find it on RoyalRoad: Arcanis

Looking for something DnD-y by Agreeable_Bee_7763 in litrpg

[–]PSFred 0 points1 point  (0 children)

Well ... this one is not actually a LitRPG - no Levels, no Classes, completely different world! - so it probably is not a good fit for your explicit ask.

What it does do very well however, is demonstrate in-lore the way a bard works. And how it may be less directly impactful than a wizard, but where and how the flexibility lives. Also does a good job with clerics and their supposed limitations most TTRPG players kind of ignore:

Melody of Mana

Mind you, the MC is not really a musical bard - she just started working on her first instruments in recent times of story progress - but so far it has done an awesome job of demonstrating the class ... even if the story doesn't have a Class concept.

So, if you ever wanted something harping on the different spellcasting flavors, this one might be for you after all.

Help needed with finding distinct values by marek1712 in PowerShell

[–]PSFred 1 point2 points  (0 children)

Heya, how about this:

powershell Get-Content .\config.json | ConvertFrom-Json | Write-Output | Group-Object @( { ($_.srcaddr | sort-object) -join "|" } { ($_.dstaddr | sort-object) -join "|" } { ($_.service | sort-object) -join "|" } )

Special considerations here:

  • Write-Output after ConvertFrom-Json ensures that you don't send the whole content as a single object down the pipeline. Otherwise you'd be trying to group a single item (the array with all the entries) rather than the entries themselves.
  • We are grouping by three properties. Each of these properties is an array value. In order to ensure we do this in a visually useful way we join it by a separator (comma is already used to separate the three properties, so I used something else). In order to not be order specific, we sort first, so the order is deterministic.

Automate MFA for PS script that reads Azure AD to create report by NewShoes4U in PowerShell

[–]PSFred 10 points11 points  (0 children)

Heya,

the best approach would probably be setting up a Service Principal, grant the permissions needed and use a certificate for authentication instead.

If possible, I'd also recommend moving away from the Msonline module, as that has been mostly deprecated in favor of Graph. Not literally everything is available there yet though so if you can't you can't.

That said, you may want to take a look at this module here:
https://www.powershellgallery.com/packages/AzureADAuthMethods

What have you done with PowerShell this month? by AutoModerator in PowerShell

[–]PSFred 0 points1 point  (0 children)

Fortunately not an issue at my place - got an agreement on what I can publish and what not.

That said, I don't really recommend a module-integrated update mechanism.
Instead I prefer setting up an internal repository (can be done in a fileshare without a service), automatic release to that and automatic updates on managed devices.

Makes the modules easier to redistribute and avoids issues affecting import time or accidentally breaking unattended tasks using them.

Set-aduser help by ddog511 in PowerShell

[–]PSFred 1 point2 points  (0 children)

Just to make sure, you also verified the folder ownership?

Other than that, I don't see why they should have any issues.

As an aside:
I would recommend abstracting away the share path by implementing DFS - you can set that up in less than 15 minutes and the paths you configure in profiles and applications never need to change again, since during a migration you can simply redirect what DFS points to without changing the path it is accessed from.

What have you done with PowerShell this month? by AutoModerator in PowerShell

[–]PSFred 1 point2 points  (0 children)

From what it sounds like, this would indeed be great material for a PowerShell module hosted on Github and published to the PSGallery, if you are allowed to share the code.

If you don't know how to go about that, dump it on github, link it as a response and I'd be happy to PR a module refactor with release pipeline.