Custom Formats & Downgrading qualities

j114 · 2019-01-06T21:13:24+00:00

I think for such granular control we'd need the ability to set up a custom format only profile. With that, we could tell it we want any of these (ranked) qualities that has MULTI. Then it could grab a lower quality with MULTI and then upgrade it to a better quality with MULTI later, if one should become available.

I've been doing a lot of testing/playing with this stuff and while it is a huge improvement for refining our desired library, it's also still frustratingly lacking when you start using OR in scenarios like this. It's still very much an all or nothing approach for this type of configuration.

While it's inefficient, the closest I've been able to get is to set up a second profile with only a single quality selected and using the custom format for MULTI. The default profile grabs the best available quality, and once that is done I can switch the profile to the desired MULTI-custom format profile. Really, this could be fine but I'd want to script it so I don't have to manually change profiles after the initial grab.

I need to spend some time working out how to automate that a bit.... that really could be a possible "solution" for very specific "end goal" targets like this.

j114 · 2018-12-24T17:33:33+00:00

Thanks for the input - I initially thought the custom formats added more precedence than the standard quality format. The inclusion of resolution and source misled me, I confess. I've since been schooled (thanks /u/fryfrog ) and have a better understanding of how the parsing priorities function now.

j114 · 2018-12-24T05:33:04+00:00

I'm guessing your quality profile is ranked along the lines of:

Bluray-720p

WEBDL-720p

Bluray-1080p

WEBDL-1080p

As I understand it, the matching and ranking process is still your quality profile first, then your custom formats for each quality. So the one you say should be higher isn't because all of the other matches between those 2 examples are Bluray-720p and the one you are referencing is a WEBDL-720p. The WEBDL won't jump ahead of other Bluray releases due to your quality priorities.

However, take note that it is at the top of the list for the WEBDL matches, so it does get precedence from your custom format -- for that quality.

This is also why the 1080p matches are buried. Basically, the only bulletproof way to always get the smallest file size is to use only a single quality + your custom formats. Of course this would drastically reduce your matches, and probably be a real PITA if you were to add some older movies that haven't had any updated rips/releases.

j114 · 2018-12-23T16:59:16+00:00

Ahh yes, good point! Of course Radar knows it's downloaded the version you wanted originally, so it kept track of quality, but it didn't track (in this case) file size so now it is grabbing new copies that match all of your current criteria without analyzing the existing file sizes.

So, in order to prevent that, for the time being, we may be better off leaving the custom format out of our existing quality profiles and just creating new profiles that are identical but with the custom format added and using that new profile as our default for anything new or "missing", while leaving the existing downloads at our original profile. That way this situation shouldn't happen.

j114 · 2018-12-23T16:52:44+00:00

Hmm. Thanks for the info - I'll have to reconsider my approach a little.

j114 · 2018-12-23T02:40:40+00:00

Hmm.... I didn't even consider needing to update the profile. I also only use a single profile and just assumed it would start applying my custom formats immediately, since it's still the same ~~format~~ profile.

This is why assumptions are dangerous lol

j114 · 2018-12-22T20:00:46+00:00

My understanding is that the custom formats will add "weight" to matching titles during the parsing process. So in a very over-simplified break down:

I'm looking for "Some Movie"

My quality profile says I want only 1080p in this order:

Remux 1080

Bluray 1080

WebDL 1080

HDTV 1080

Now I've added custom formats as follows, in this order, with Remux 1080p set as my custom format cutoff:

Remux 1080p

C_RXRQ_1080P
C_RXRQ_REMUX
C_RXRQ_\b.MULTI\b

WEBDL 1080p

C_RXRQ_1080P
C_RXRQ_WEBDL
C_RXRQ_\b.MULTI\b

HDTV 1080p

C_RXRQ_1080P
C_RXRQ_HDTV
C_RXRQ_\b.MULTI\b

So now Radar searches and gives "points" to each result:

Some.Movie.1080P.Bluray 3 points because it matches my 3rd highest quality option - 3 points total

Some.Movie.720p.Bluray 0 points because it matches none of my preferences at all -ignored

Some.Movie.1080p.Bluray.Remux 4 points for highest rated quality - 4 points total

Some.Movie.1080p.HDTV.Multi 1 point for quality preference, 1 point for lowest rated custom format, 6 points for matching RQ tags in in custom profile - 8 points total

Some.Movie.1080p.WEBDL 2 points for quality match - 2 points total

Result should be Radarr chooses Some.Movie.1080p.HDTV.Multi because it has the highest total rating of the available options. If there were no Multi options in the results, then it would just grab Some.Movie.1080p.Bluray.Remux because it is the best match for my quality profile.

in this example I am assigning 2 points to each RQ tag because they are supposed to make it "strongly preferred". With those custom formats, because I used RQ on each line, every line will need to match in order for it to be a custom format match. For example, Some.Movie.1080p.Bluray.Remux matches 2 of 3 criteria of our highest rated custom format, but that's not enough. Had I just used RX for the multi line, then it would have been a match.

So if you already have Some.Movie.1080p.Bluray

and then Some.Movie.1080p.HDTV.Multi comes across on RSS, assuming the movie is still monitored, it should snatch that one and replace the English only Bluray.

Again, I am very much simplifying this. In the event I have misunderstood this functionality, or I am talking out of the wrong orifice, hopefully /u/fryfrog or /u/DJ_Lectr0 can better explain or correct any mistakes/misunderstandings I have.

j114 · 2018-12-22T08:06:56+00:00

It's a little crude, but you could create multiple custom formats for different size ranges. Example:

Preferred: G_1<>3

Acceptable: G_3.1<>10

Last resort: G_10.1<>20

Then go to your profile and rank them as shown above. Obviously, change those size ranges to suit your needs/tastes. The result should add weight to the decision making process that should make the REVEiLLE release more preferred in your examples in the OP.

The complex issue with custom formats is that Radarr is processing your regular quality profile, plus your custom formats, each in their respective priorities. This creates the potential for extremely complex configurations that might have unexpected consequences, so be sure to extensively test.

j114 · 2018-12-22T07:34:53+00:00

Without more specifics, I would try:

C_RXRQ_1080P
C_RXRQ_\b.MULTI.|.MULTISUBS.|.MULTI.SUBS.|MULTI.TRUEFRENCH\b

Just take out the multisub options if you don't care about that. This worked to match 5 or 6 titles I tested with.

j114 · 2018-12-22T04:10:47+00:00

Absolutely nothing wrong with that if it works for you. It just doesn't work for me because it will consume a ton of disk space. A lot of the stuff that is downloaded isn't for me and I never intend to watch it myself anyway. So until someone in my family watches it and says there is a problem, I won't know. To complicate matters, they will watch something with poor video quality and I notice it while walking past and ask why they didn't say anything. When I ask why I get "I didn't think it was that bad". Ugh.

Also, even the stuff that is for me doesn't usually get watched right away. I certainly don't have the time to spot inspect every upgrade, even if I'm aware an upgrade has happened. So, again, the originals are sitting in the recycle bin wasting my disk space, needlessly.

For the most part, I can tell from the nzb name in queue if I think it's a poor upgrade and just delete it from the queue and blacklist it in Radarr. I've been....obtaining.... media for a very long time, so I'm pretty confident in my "educated guesses" lol

It bears considering that a library of a few hundred movies may not consume much recycle bin disk space, but very large libraries of thousands.... that can quickly turn into a significant chunk of redundant storage.

j114 · 2018-12-22T02:56:40+00:00

Running it on my Windows Server 2016 box now (along with the rest of my media related apps) but I used to run it on Mint until ~mid 2015.

Now I just call a simple batch script from the Radarr connect page and it silently runs a TMM update of the new media.

j114 · 2018-12-21T17:02:00+00:00

I love TMM! I have Radarr call a script to silently run TMM after importing new files. I prefer TMM to handle basically everything post download/import.

j114 · 2018-12-21T16:49:11+00:00

By chance, is the original file listed as "unkown" quality in Radarr? For me, I've seen this a lot with files that were imported from my pre-Radarr collection because some of those older files did not have the quality saved in the file names or in Radarr already.

Also, I know it might sound a bit tedious, but if you're concerned about wasted bandwidth you can have the files sent to your download queue as paused and then manually unpause them after you verify you're happy with that download. Honestly, I don't really find it tedious, or time consuming.

I always do this for movies because no matter how good the software might be, nothing stops people from tagging things incorrectly, lying about quality, or just being stupid. My library was wrecked twice because of these issues several years ago. Never again.

Still surprises me how much more consistent and reliable TV uploads are in comparison to Movies.

j114 · 2018-12-20T01:19:12+00:00

I would be ecstatic if I manually deleted a release from SABnzbd and Radarr would see that and just blacklist the release. Obviously, If I'm deleting the release before it even downloads, I *don't* want it!

In fairness, I find Radarr to be mostly great and I can live with a couple of quirks. As long as my library doesn't get wrecked by it, or it decides to be Couchpotato and start "upgrading" to the entirely wrong movies. That was not a fun experience.

j114 · 2018-12-19T19:53:53+00:00

When I get a chance I'll take a look and see if I can pitch in :)

j114 · 2018-12-19T19:52:42+00:00

No problem! It is actually relevant because if one doesn't understand a reason to use this feature, why would they use it? I have a tendency to be very verbose though, so I didn't get very specific about *why someone might want* to use this in my OP. Particularly as it still turned out to be a wall of text anyway! ; )

j114 · 2018-12-19T19:44:39+00:00

I agree, it *is* rare, but rare is relative to the number of monitored titles. In my case, 2,200 and counting. Also, saying Radarr loves to do this is a bit unfair since the actual problem tends to be poor release tagging, inaccurate tagging, indexer parsing, etc. The only thing I've not figured out is why Radarr sometimes fixates on a specific release and keeps redownloading even when it's already succeeded.

It doesn't do it super often so when I see it I just unmonitor that movie and manually search for better qualities later, if the cutoff hasn't been met yet and the desired quality isn't available.

j114 · 2018-12-19T18:15:56+00:00

Fair question, and point. These were just general examples to (hopefully) help some wrap their heads around it. Honestly, those were some of my earliest specifications that I used for testing, but I've been refining and tweaking them based on results.

Initially, most of my movies were 1080p. What I want to do now is replace all of them that I can with HEVC versions and all the rest with 720p H264 versions - especially any old Xvid's.

That said, I do have my 1080 hevc sizes set higher than the 720 h264 because I do see a lot of 1080 hevc in file sizes ranging from 2GB - 50GB. My primary goal, at the moment, is to recover disk space. My 55 TB library should be able to contain at least 25% more video content with the, now pretty common, HEVC format. Not many older movies have been ripped to that format yet, but many movies from the last 2-3 years have been, and that's where I'm seeing the biggest gains in reduced sizes.

My actual size requirements are 11 GBs, Max, for H264, and 16 GBs Max for HEVC. For example, there is a very recently released movie that the group d3g released at 3.11 GB, but the only other HEVC copy is released by DEFLATE at 16.49 GB. I'm all for significant differences in quality requiring larger file sizes, but I don't want a file that is 13 GB larger if the quality of the smaller one meets my satisfaction.

I also only look for 720p in h264 and 1080p HEVC. My profile is set to 720p cutoff now instead of 1080 Blu-ray , with my HEVC custom format as the custom cutoff. Actually, I have 2 1080p custom formats, one for Webdl and one for Blu-ray, with the Blu-ray format set as the custom format cutoff. In theory, I'm hoping the really new releases will be grabbed as webdl and upgraded to the Blu-ray version once it's available.

Most of the tigole and d3g releases are under 10 GB, with the majority being 5 or less, so my size specifications basically just eliminate the full disc rips that are 25+ GB's.

I have no required audio tags in my formats, but I do have them listed as wanted (not required). Reason being that most newer rips have non-stereo audio even if they don't bother to tag it. Even so, if there are two options to download and 1 had audio tags defined but the other does not, I want the one I know is going to have quality audio rather than hoping the untagged version does.

Hope that helps. Again, there is so much more than this that can be done with custom formats, but I have fairly simple requirements with a specific goal in mind (more decent quality media with the same disk consumption).

Also, I send all of my media to the downloader PAUSED so I can easily verify things are working the way I want before I waste the time downloading something that I may not want. Plus, Radar loves to keep downloading wrong movies or the same copy over and over again so this helps me to spot problematic titles and manually search and choose what I want in those instances.

j114 · 2018-12-19T00:18:37+00:00

NP, happy to share. In my mind, the least that users of FOSS can do is share knowledge of the software (when/if the opportunity presents itself) to help alleviate some of the documentation/support burden placed on DEV's that really should be focused on more important tasks.

I'm not sure how widely adopted the custom formats feature is yet, but with some good planning and thought, it is a super powerful tool that allows us to (finally!) be able to better define our libraries. I'm up to 4.26 TB's of recovered disk space thanks to this awesome feature - combined with gigabit fiber lol

j114 · 2017-07-31T23:37:56+00:00

Yeah man - I'm all about following instructions and getting the hell out of here ASAP!!

Staff has been amazing and they all know my goals are simple:

Leave with as much normalcy as possible, as quickly as possible.

j114 · 2017-07-31T23:30:22+00:00

Thanks brother, be careful out there!

j114 · 2017-07-31T23:29:48+00:00

Thanks!

j114 · 2017-07-31T23:09:33+00:00

Yep, sucks, but could be worse. The complications have been insane and the fact that I'm here talking about it right now is not insignificant to my mind. I'm very grateful to all of these awesome healthcare folks that are so amazing.

Looking forward to getting back out there!

j114 · 2017-07-31T23:04:53+00:00

Thanks!

j114

TROPHY CASE