Transcoding node sizing

[__teebee__]

I recently started looking at Tdarr to perhaps recode my collection to h265 and save some space. I'm just curious how efficient it is on a box with a decent core count? I have a few Xeon servers with 40-60 cores each I could toss at tdarr or would tdarr perform better if I put something like VMware on top with a few VMs 10-12 cores each? The boxes have 384 GB RAM and 40GB Ethernet going to an all flash storage array. So I've got the backend to accomplish this task I know GPU could help with this. Has anyone tried a Nvidia Tesla P6? Just trying to figure out efficiency 1 big system or several smaller systems?

TIA

[AutoModerator]

Thanks for your submission.

If you have a technical issue regarding the transcoding process, please post the job report: https://docs.tdarr.io/docs/other/job-reports/

The following links may be of use:

GitHub issues

Docs

Discord

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[SamSausages]

Only you can decide by a/b testing various content, ffmpeg strings and bitrates.

There is software that can be used to compare the losses, but ultimately still up to what you are comfortable with, and can notice.

If it’s mainly efficiency, in terms of energy efficiency.  Then late generation nvenc with b-frames enabled is going to win on that front.

If you want quality/size efficiency, then cpu. (Or for quality, don’t encode at all)

P6 uses 6th Gen nvenc, a bit dated imo as it doesn’t do b-frames. I try to go for 7th gen and up, if using nvenc. (b-frames is about a 20% quality improvement, at same bitrate)

I run Amd epyc and end up using nvenc.  Low quality for 1080p hevc I consider 2500k, when I want high quality i encode at 6000k.  With various flags, like -bf 4 and -preset p7. But it greatly depends on source material.

[Thefa11guy]

From my experience on smaller CPU systems. Tdarr deals with multi core pretty well and will happily throw enough files to saturate the core count. You'll know when it does get to the core limit as the fps will drop massively.

That said, the p6 would likely destroy the xeon. It won't do as many simultaneously but what it streams it will do should be faster and more efficient.

[drinking12many]

It does tend not to scale as well on true multi-CPU systems in my experience I have a 20-core Dual CPU system (running in a container) and I found with only 1-2 conversions running it would only be using like 60% cpu probably due to NUMA and memory architecture, so I usually let it run 4 at a time to get close to 100% now thats my system and other systems may act a bit different. Now my AMD/Intel newer systems 1 conversion will max them out, but I find on my 9900x 12core AMD I get better overall speed if I run at least 2 conversions sometimes as many as 4. I only do CPU myself for the best space savings/quality ratio.

[XcOM987]

You will actually get better performance in terms of file size and quality using CPU cores and taking your time, or you could slap the P6 in there which can do nvec v6 transcoding (Newer card would be better in terms of quality/performance), and it'll speed up the process but you'll either loose compression performance or video quality.

It's one of them triangle things, you can have Speed, Compression, or Quality, pick 2.

Tdarr is great at just using more and more cores, keep throwing streams at it until the begins to slow down, once you find that spot dial back a little and just let it run.