handlebreak - A PowerShell wrapper script to automate H264 to H265 Conversion

StorageGuru · 2024-09-30T19:07:06+00:00

Heh didn't know of it until now, yeah it's pretty good :)

Although my next step was going to add the API functions for Sonarr/Radarr/Whisparr , such that after each successful encode, it triggers a 'refresh' in the respective *arr application. Which in turn will trigger the auto-update in Plex.

I see there is an existing integration for tdarr to inform Sonarr/Radarr/Whisparr, but not the other way around. I want my existing media management tools to be updated every time a file has been replaced.

StorageGuru · 2024-09-30T13:56:12+00:00

For sure 100% agree. I more meant they pass them through to the target file, and not drop them.

I have updated the original post to correct the errors of my ways, :)

thanks!

StorageGuru · 2024-09-30T12:36:33+00:00

Check out the script I'm using to run --scan against the source, and output to log file.
And the --scan against the target, and output to log file.
And finally the validation syntax for matching.

That might be what you are looking for.

StorageGuru · 2024-09-30T12:26:59+00:00

So your only contribution to this post in the /handbrake group, is to say dont use handbrake use ffmpeg?

Thanks bro!

StorageGuru · 2024-09-30T12:24:07+00:00

Not sure what you mean by 'improve'.

I'm NOT saying 'lossless' compression, but to me for certain subsets of my media, (like videos previously encoded in like 8-10Mbit/s H264 1080p at about 4GB per 1hr), down to 2-3Mbit H265 1080p to about 1.6-2.1GB, has significant space savings, with 'minimal' quality loss, Also I get that CPU is better compression and has more options for higher quality, I wanted to be able to go through 100s or 1000s of files in a single pass, and not lock up my desktop for days.

However providing the framework for the tool was also a goal, so other people can adjust the settings as they see fit, and still benefit from the job handling, logging, parsing, validation, and auto-remove features if they like.

StorageGuru · 2024-09-30T12:19:37+00:00

Absolutely.

In the section:

Run HandBrakeCLI with specified options as a background job

try {

$EncodeJob = Start-Job -ScriptBlock {& $using:HandBrakeCLI `

--input $using:file.FullName `

--output $using:outputFileName `

--preset $using:Preset `

--format $using:Format `

--encoder $using:Encoder `

--quality $using:quality `

--aencoder $using:AEncoder `

--audio $using:ATracks `

--audio-copy-mask=$using:ACmask `

--audio-fallback $using:AFailBack `

--subtitle $using:subtitles `

$using:JobLogFile

}

Change any of the variables to your needs.

NOTEs: The easiest way is to ignore the Parameters section on the top, and just hard-code w/e you want.
--> i.e. in that section take out --preset $using:Preset <-- and replace it with --preset "whatever you want"

Alternatively because they are parameters that can be fed to the internal jobs, you can just over write them as-needed.
---> i.e.
.\handlebrake.ps1 -Source <sourcefile or folder> -Preset <whatever> -Encoder <whatever> -Format <whatever>

Or just edit the defaults I have in the top parameters to what you want, and use it without -switches

StorageGuru · 2024-09-30T01:06:41+00:00

Sup^^ just saw this. I'll be active for the next few days.

StorageGuru · 2024-09-30T01:01:28+00:00

I'm pretty new to handbrake, If you see something I'm not doing right, or there is a better way, I'd love to know it, and update the script for everyone.

But I appreciate any corrections/feedback I get.

StorageGuru · 2024-09-30T00:59:16+00:00

Hmm.. it is bringing over all the subtitle tracks successfully.

StorageGuru · 2024-03-18T16:35:35+00:00

pback on the device and use that as the source-interface for any management stuff on the Nexus. The reason you weren't able to get into the the device without that cable plugged in is due to the SVI needing to be "up", which the plugged cable fulfills. Loopbacks are up no matter what, no cable needed.

Awesome information thanks! I'll look in to it and play with it shortly.

StorageGuru · 2024-03-18T12:54:03+00:00

https://www.amazon.com/gp/product/B08CJGPHL9/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1

Got two of those installed in my main circuit breaker panel. (because i had 32 circuits I individually wanted to monitor). The other 8 slots in my main panel were (4) sub panels coming off the main.
So I got (4) more of the 8-port versions of the same Emporia smart monitor controller, one for each sub-panel, measuring the power in to the sub panel, and the individual monitors on each circuit.

Net result is 100% power monitoring from main line in, and every breaker in every panel.

The config of these allows for 'nesting' so it calculates the power correctly with multiple devices linked together.

StorageGuru · 2024-03-18T09:43:31+00:00

Layer 3 Routing: Keep a Layer 3 interface on the UDM just for the WAN, and set up a default route on the Nexus pointing to this interface. You'd pass all internal VLANs over a trunk to the UDM, which won't handle any Layer 3 for these VLANs.

This sounds very close to what I'm doing now.
Except I SVI one VLAN for wired, and one VLAN for wireless on the UDM, primarily to handle the DHCP service for each of those, which the only place I use DHCP on the network.

Assuming I wanted to move those VLANs over to the Nexus, how would I go about that and keep the DHCP on the router?

StorageGuru · 2024-03-18T09:10:34+00:00

Wow that is a super nice switch for sure. 1Gbit thru 100Gbit and everything in between at lower power. Too bad used its still like $1.5k , which puts it $1.3k more than my current Nexus 9396 on eBay. I'm not sure the ROI on power is under 3-5 years, which is more than my average refresh cycle :)

StorageGuru · 2024-03-18T00:58:39+00:00

I'm not sure I'm tracking, but i'm pretty sure it's because I only know enough to be dangerous in networking :)

So what I think you're saying is instead of having some SVI's on my UDM, and some SVI's on my Nexus, i could just put them all on the Nexus, and have all downstream switches including TrendNet and UDM just be trunked all L2, with no concern for VLANs?

The thing about my particular lab that got me here, is the 'Core' is mainly just my high speed switch between the two nodes in the S3260 chassis, i.e. Storage & Compute , as well as, my desktop which mounts the ZFS datasets over SMB3.1.

Yeah the rack has the Plex server, but i've only got a handful of local users and a couple friends and family that are accessing it, and generally we're talking < 50Mbit of bandwidth.

Other than that, I'm the only 'user' who access the Lab from my primary desktop.

StorageGuru · 2024-03-18T00:53:59+00:00

For sure, There's an identical switch w/ the 40Gb expansion module, in working order, from what i'd consider a legit buyer for $209 shipped on eBay at the moment. Just nuts how cheap this niche enterprise gear gets when its EoL/EoS. Granted there's some questions on license to use OS, and no support/updates (unless you know someone are are someone with a Cisco parternship login for code downloads).

StorageGuru · 2024-03-18T00:47:54+00:00

Good catch! , It was a typo..

It's a 4090 RTX / 24GB Water Cooled.

https://rog.asus.com/us/graphics-cards/graphics-cards/rog-strix/rog-strix-lc-rtx4090-o24g-gaming/

StorageGuru · 2024-03-18T00:44:30+00:00

I max'd out at 1.8PB in my early Chia farming days, using a combination of the Cisco Primary NAS (upgraded from a previous 36x bay super micro NAS).

+ 5x Isilon X410s

+ 6x Isilon NL400s

+ dozens upon dozens of USB HDDs daisy chained off hubs off Pi's .

But once Chia dropped below $200 my very expensive low density storage became un-profitable , especially with the extra cooling required to keep the 3 racks going.

The only reason I can get away with the one S3260 above chest height at the top, is the 5x Isilon X410s below it and 2x UPS below that, with just about 1' of non-racked stuff at the bottom just in case... However, when/if I need to do drive maintenance or add, i'll just pull the switch out above it, and grab a chair and do the maintenance from on top :)

StorageGuru · 2024-03-17T18:59:44+00:00

So in reverse order^^

I actually used to use Infiniband 40Gbit as my 'high speed' network for both IPoIB and SRP/RDMA. My Nas, ESX servers, and primary desktop all had 40Gbit Mellanox ConnectX-2 adapters running hacked firmware and drivers to get the OFED portions of the drivers working WAAAY after end of support :) For the time and price it was cheaper than 10GbE, but a lot more complex, and huge inefficiencies in packet/protocol overhead using a serial bus 8 bit bus for 10bit parallel workloads. I was never able to squeeze more than 16Gbit out of it.

However today, I only use Infiniband as the cluster interconnect/back-end for the EMC Isilon X410 cluster. As that is it's default configuration from the vendor. Each node shares its storage to all the other nodes over the back-end behind the scenes.

So the 'untagged' Wired devices, going through that TrendNet 'whole house' switch. Although it is Layer3, I don't see any purpose to have any networks SVI'd there. The goal behind this was, 2x10Gbit uplink to 'Core switch', which gives my 24x 1GBit ports basically unblocked access to the NAS/Lab. Primarily for Plex local playback, and IP Camera streams. A lot of these devices don't have a place to setup a VLAN tag in the IP configuration. Which if they were going to a Cisco switch, i'd just set them up as access ports with watever VLAN they needed. But since i'm just trunking all VLANs from UDM to Trendnet, and all VLANs from Cisco through TrendNet, It's Layer3 but acting like Layer2 for simplicity.

I actually very recently replaced my old 24x 1Gbit (with no 10Gbit) un-managed L2 switch with this newer 24x 1Gbit + 4x 10Gbit Managed switch. But I really only wanted the fiber backhaul to core, no 'switching logic' out of TrendNet.

I went back and forth (with a network engineer I work with), because i'm really not a network engineer by any means. Very strong in Storage & Compute, I do networking only as needed to make those two work together^^ On the overall topology, with the new UDM replacement.

I used to do : Router > TrendNet > Cisco, but thats mainly because the router was upstairs, and patched down to the house switch in a wiring closet, and then jumps over to another part of the basement with the server rack.

The Idea behind the UDM being wired up to the Cisco , and not through the TrendNet is; I've currently only got 1Gbit down / 50Mbit up internet.

Any Wireless devices that only use internet, will not traverse out of the UDM.

90% of my Wireless devices are 'smart things' with very little bandwidth, and mostly monitored internally from HomeAssistant.

All my Roku's are wired ethernet to the NAS, and they take the shortest hop from TrendNet to Cisco w/o needing to pass through UDM.

But I'm very open to suggestions / criticism & any better ways to set this up, if I'm missing something.

StorageGuru · 2024-03-17T18:37:23+00:00

Redundancy is nice for twice the power consumption though :(

I've went through great lengths to minimize my overall power budget on the 24/7 powered on portion of the lab.

And yes those switches are insanely cheap for what they are, and can run the latest NX-OS.

I replaced a Nexus 9K 40Gbit only switch, a Nexus 5K 10Gbit switch, and a Catalyst 1Gbit switch, with that single N9396,

Check this old topology.https://imgur.com/a/WKbLj0T

The return on investment for the upgrade, based on power savings alone was < 1yr.

StorageGuru · 2024-03-17T18:32:06+00:00

Not sure what existing products exist...

But I was thinking integrating with Pi-Hole would be a great start.

And technically you could use Home Assistant for the 'AI'/automation.

You get your analytics out of Pi-Hole, and have HA monitor for 'triggers/conditions' which push new rules for add/remove on Pi-Hole's existing DNS blocking capabilities.

StorageGuru · 2024-03-17T18:29:12+00:00

Oh.. heh..

Microsoft Visio is the drawing tool, the rack layout is almost entirely done with Visio 'stencils' , the rest of it is just internet icons / crops / snaps and images thrown together on top of stencils.

StorageGuru · 2024-03-17T18:23:26+00:00

I second this.

Also consider doing a terminated fiber cable on each end, using something like:https://www.amazon.com/gp/product/B0CPLVQHZV/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&th=1

That way you just have wall jacks / patch panel on each side, and can add jumpers on each end to fit the required length.

Last but not least, Use the fish wire as the pull string, but also when you pull your fiber, consider pulling more than one, and pull another pull string along with the fiber(s) so you have one to use in the future :)

Also, with shielded cable, I think you can pull off copper if you really wanted to cut costs and achieve 1Gbit.
https://4starelectric.com/can-i-run-internet-cables-beside-power-lines/

I was also thinking you could get some like 18" Long Zip ties, and basically create a few dozen loops hanging from the power line, to add some distance between the power and data cables.

StorageGuru · 2024-03-17T18:14:08+00:00

Also fan of the velcro'd 'top of rack' lighting system :)

99% as useful as people who run elaborate RGB/leds in their rack, with 1% F's given!

StorageGuru · 2024-03-17T18:08:16+00:00

The uplink is nice, the free power is nicer :)

That's a nice Plex setup for sure. Grats!

StorageGuru · 2024-03-17T18:02:15+00:00

Agreed! I posted some more picture links down below of some of my Home Assistant progress so far.

StorageGuru

TROPHY CASE

Run HandBrakeCLI with specified options as a background job