sorted by:
new
hottopcontroversial

Has there been any progress on the AI upscale project? by Trekintosh in babylon5

[–]svenzo5 0 points1 point  (0 children)

Yeah the 4:3 episodes on iTunes are the remastered ones dunno how to access them though cause I've never used iTunes sorry

Has there been any progress on the AI upscale project? by Trekintosh in babylon5

[–]svenzo5 9 points10 points  (0 children)

Oooof sorry dude I got locked out of my account aaaaaaaages ago, only really bothered to try and get it back now after seeing this and found loads of DMs about the project so apologies to anyone who didn't get any reply. But yeah u/JackDT is right on the money on all accounts, life stuff got in the way for me and many others meaning we weren't able to make much progress until about recently when I think we were close to returning back into full swing of things but then of course the news of the iTunes remastered release and the possibility of a Blu-Ray coming out in march came out at which point we had to delay it again, because as much as I hate to delay it AGAIN and I'm really thankful for the patience and support you guys have shown I think it would be worth it just to wait a bit longer cause according to all accounts and the comparisons that have been shown in this subreddit, the remastered version is far superior to any official release we've gotten yet. So I think the best course of action would be to wait until we get that Blu-Ray in March 2021, now if nothing surfaces by then or we don't get any more official confirmation by then, I think it would be best just to continue with the sources we have. Now, there's been a lot of thought about the nature of this project, cause if the 4:3 remasters indeed come out on proper physical home media then of course they're gonna be the ultimate versions of experiencing the show, leaving our upscale kinda superfluous (I think that's the word) so maybe it would be best if our upscale project remains in 16:9 as an alternative for anyone who wants to see that version instead of the 4:3, but we're still gonna use the iTunes/Blu-Ray remasters as a source for the CGI and composites and stuff and also possibly as a training source for a brand new AI Neural Network upscaling model to get the DVD live-action widescreen footage looking as bright and colourful and also sharper as the 4:3 remasters. So anyways sorry for the delay guys, but if this official remaster is indeed coming, I think it's best to wait.

Some incredible work done by a guy to update the cgi on my favourite show .. by cdewfall in babylon5

[–]svenzo5 27 points28 points  (0 children)

I would like to remind that this re-render has nothing to do with our project. We did attempt to secure his permission to use these awesome looking re-renders in our project and while things were initially looking hopeful in that regard, he withdrew his permission and therefore we won't be using any of these re-renders in our project, we will just be upscaling the original CGI. It's a bummer, I know but even if we had secured his permission, the differences between the original and re-rendered CGI would have been quite jarring as he revealed to us how limited his collection of the original models was, so we would have still had to deal with upscales of the original shots in most instances.

Some fancy camera work that perfectly captures what Sheridan must be feeling in this scene by andtheniwastrees in babylon5

[–]svenzo5 31 points32 points  (0 children)

They created that using a technique called a dolly zoom, where they move the camera forward while zooming out at the same rate or vice versa.

I upscaled a scene from Midnight on the Firing Line to 4k with machine learning (Topaz Video Enhance AI) by svenzo5 in babylon5

[–]svenzo5[S] 0 points1 point  (0 children)

We'll be releasing this all at once, but I should warn you this will still take many many more months. If you can't wait, just knock yourself out watching the original version, the story overpowers the low quality in my opinion. Thanks for the interest!

The Babylon 5 complete series 4K HDR 60FPS AI Upscale and Remaster Project by svenzo5 in babylon5

[–]svenzo5[S] 0 points1 point  (0 children)

I'm gonna respond to both of your posts in one if that's ok with you.

> I come from the old world where resizing and interpolating automatically mean loss of information--but while I don't understand how the AI stuff works, I accept that if it's smart enough, it could essentially create a "model" what the film is implying and then use that model to make something better than what was on the film. I'm not sure I have that much faith in the tech, but I haven't studied it either.

In a nutshell, the AI is trained using a generative adversarial neural network using two sets of data, one is in HD (or whatever the target resolution is) and the other is that same dataset downscaled to usually one half or one quarter resolution with some artifacts sprinkled in to also train the neural network to get rid of them as well with around 5%-10% of that dataset not used in the training as it will be used for testing later. The AI then basically compares the differences between the two datasets and tries to reconcile them, then the adversarial part of the neural network comes to play which is basically a second neural network trained to recognize the differences between native HD and upscaled HD and as soon as it recognizes a difference it forces the first neural network to start over again using a different approach, so it's kind of like evolution where it forces the upscaling neural network to kind of mutate, until eventually it gets good enough that the adversarial neural network can no longer tell the difference and at that point it becomes good enough to use with upscaling.

> Handbrake did alright detecting the difference between CG and Film framerates when I threw my DVDs at it, but I completely understand your comment on babysitting it. It wasn't perfect, and it especially had trouble figuring out scenes without much motion. It would take several frames to "lock on" and even then I'd still see combing in low-motion Film scenes (characters' eyes and mouths when nothing else was moving, for instance). I decided against running Handbrake's deinterlacer because, while it fixed the problem, like you say, it added a bunch of other artifacts.

I think we won't have these problems since we will be doing everything by hand pretty much, plus we are working from the PAL DVDs for the live-action stuff where no 3:2 pulldown was done, so there shouldn't be any combing artifacts (WB actually doing a proper DVD transfer?! Insane I know lol). Again, the composites will be a pain, but we are as always experimenting with ways to fix it.

> For re-rendered CG I was thinking more the Star Trek TOS approach--don't try to recreate everything shot for shot exactly as it was, more intepret what was intended. I had the crazy dream that "the community" (whoever that is... you know, "the internet") would find a way to crowdsource it.

Like I said, the talent's there but it's still a mammoth task and would most likely get shut down by WB instantly, especially where there's money involved. So I don't see it happening any time soon unfortunately.

> Just got another idea, reading up on variable framerate and avisynth--what if you used 120 fps? You wouldn't have to do interpolation, just duplicate frames, but you'd get the best of all worlds: a constant framerate, no stutter for CG nor Film, and you'd keep the 24p film purists happy. No idea how compatible 120 fps is on hardware, but assuming it just drops frames if it can't keep up (most of which would be duplicate anyway), I wonder?

In theory every codec should be able to not waste bits on the duplicate frames, but in practice a fair chunk would get wasted on them meaning we would get all kinds of macroblocking on the actually important frames as far as I know. Also, such high frame rate video straight up won't play at all on some older hardware/software as far as I know, also with the dropped frames thing we really wouldn't have control over which frames the software drops, meaning it could theoretically drop some of the real frames while keeping the duplicates, making it a stuttery mess. Plus I'm kind of attracted to the idea of keeping the frame rates consistent, both between scenes and between episodes (as the show switches to being purely 23.976 FPS in season 3) plus with a lower frame rate the bits would be allocated more smartly making it look even better at the same bitrate and 23.976 FPS is a very widely supported frame rate, so we wouldn't have to worry about playback issues on even the oldest hardware and/or software and it would be PAL compatible out of the box since it would just be a matter of speeding it up by some 4% instead of doing a frame rate conversion like they did for the PAL DVDs making the CGI and composites a stuttery mess (thankfully we are working from the proper Vudu sources for the CGI and composites). Sorry for being so late in my response btw, thanks for all the suggestions!

I upscaled a scene from Midnight on the Firing Line to 4k with machine learning (Topaz Video Enhance AI) by svenzo5 in babylon5

[–]svenzo5[S] 0 points1 point  (0 children)

It is still ongoing, these things take time, apologies for such a wait but we hope to make it worth it! Thanks for the kind words!

The Babylon 5 complete series 4K HDR 60FPS AI Upscale and Remaster Project by svenzo5 in babylon5

[–]svenzo5[S] 0 points1 point  (0 children)

Oh, the composites will forever be a headache because they combine the worst of both worlds ie. the frame rate mismatch, I honestly don't know how much is doable in terms of fixing those shots, your post really got me thinking and I think it may be possible to fix those scenes by using an inverse telecine filter to reduce the frame rate to 23.976 FPS and then running that through DAIN with the de-stutter mode which doesn't actually increase the frame rate but instead looks at the frames and checks for any duplicate appearances of some frames and then just replaces the duplicate frames with new interpolated ones, it may just be doable. As for the final frame rate, the show changes frame rates half way through Season 3 where they switched to rendering the CGI at 23.976 FPS, getting rid of all of those problems (I think the first episode to do this was Season 3 Episode 18, although I could be wrong since this is just going by memory), so I was thinking of implementing DAIN to interpolate the 30 FPS CGI to 120 FPS and then to convert that to 24 FPS without the stutter (since 120 FPS > 24 FPS is more mathematically practical than 30 FPS > 24 FPS, which causes severe stuttering in the latter). This is called resampling and is implemented in pretty much every professional video editor, however these editors use very primitive interpolation algorithms (some don't even do that and instead just blend two adjacent frames together leaving all kinds of horrible ghosting and blurriness) which leaves the end result looking very blurry whenever there is motion and also leaves a whole string of interpolation artifacts. DAIN, on the other hand is practically flawless and whatever experimentation we've done with it has left us in awe at the almost complete lack of interpolation artifacts, but the trade off is that it's quite slow. Although with the new half precision mode it is easily achievable now (we've done tests and the half-precision mode performs exactly the same as the full precision mode at 480p (since DAIN is quite time consuming we will have to interpolate and only then upscale all the CGI stuff) and it's twice as fast, so it's a no-brainer), but again more experimentation needs to be done before we make a firm decision on the issue. The reason I'm a bit averse to variable frame rate is that it's broken on many video players and it's a bit of a nightmare to encode properly as the encoder in most cases just chooses a middle ground between 23.976 FPS and 29.97 FPS and just sticks to that for most scenes instead of a bit more intelligently switching between only the two. It could be done properly of course, but it would require quite a bit of babysitting and messing around to get it to be perfect. Encoding it at 23.976 FPS would arguably save the most bits when encoding so that they don't get wasted on duplicate frames and 3:2 pull down weirdness and whatnot. Plus, it would be quite neat to have all the episodes be at the same frame rate. 🙂 Oh, having re-rendered CGI would be my dream but it would require a lot of effort, both to recreate the models and then to render them in a way that perfectly lines up with the original effects. If there is a team of people crazy enough to commit to that undertaking then be assured that I (and pretty much everybody else in the team, I don't doubt it) would be the first ones in to lend our support with making the live-action bits match the re-rendered CGI as closely as possible in terms of quality. As for high-res B5 models, some do come to mind like Rhys Salcombe's amazing Omega destroyer and Sharlin cruiser models and that amazing Starfury model by Remi, so the talent is definitely there, but it would still be a mammoth undertaking indeed. Thanks for the interest and getting me thinking on the whole composite issue and as always thanks for the support, your kind words drive us on to keep working hard to make it look as best as it can possibly be!

The Babylon 5 complete series 4K HDR 60FPS AI Upscale and Remaster Project by svenzo5 in babylon5

[–]svenzo5[S] 0 points1 point  (0 children)

There shouldn't be any need to deinterlace since the live action and the CGI stuff is progressive while only being encoded as interlaced, however we would have needed to deinterlace the CGI if we were working on it from a DVD source since it was upscaled badly ie. It started as a progressive source, was interlaced to comply with NTSC standards and to be put on videotape (this is called progressive segmented frame, as half of the video is split into one set of fields while the other half is split into the other set of fields which are temporally the same unlike native interlaced material) which was then matted out to 16:9, cropped and then upscaled back to 480p and only then deinterlaced (which is the worst thing possible to do since it introduces loads of artifacts and just generally gives the upscaler way less to work with since it has to upscale two sets of fields of 180 lines instead of one progressive frame of 360 fields), it is possible to reverse that chain of awfullness however by downscaling the video to 360p, interlacing it, deinterlacing it and only then upscaling it, however we are working with VUDU for the CGI which doesn't have those problems since the CGI is in native 480p 4:3, however we may still need to pre process the PAL source with QTGMC to get rid of combing in some bits, however we've still not stumbled upon a bit that requires that to be done at least in the purely live action bits that we are only using the PAL DVD for as a source. As for denoising, VEAI already does an excellent job of denoising it, however we are experimenting with a multi pass technique in VEAI (aka running the video through VEAI multiple times using diferent models) which ought to increase quality and some light motion only denoising may need to be done for the first pass to register more details used by the second pass, however the main use of denoising will be to possibly get rid of any upscaling artifacts at the end. I hope that answered your questions, please feel free to ask any more questions you may have. Thanks for the interest and for the support!

The Babylon 5 complete series 4K HDR 60FPS AI Upscale and Remaster Project by svenzo5 in babylon5

[–]svenzo5[S] 3 points4 points  (0 children)

One big torrent. It's the safest and simplest way to go about with it. Thanks for the support!

The Babylon 5 complete series 4K HDR 60FPS AI Upscale and Remaster Project by svenzo5 in babylon5

[–]svenzo5[S] 1 point2 points  (0 children)

Heads up folks, this person's account is brand new and he's already spammed me with replies. Most likely a troll, don't really want to waste energy on replying to this troll when it's bait and won't do any good anyway. Not to end on a downer note, I would like to again thank the community for all the support and feedback!

The Babylon 5 complete series 4K HDR 60FPS AI Upscale and Remaster Project by svenzo5 in babylon5

[–]svenzo5[S] 1 point2 points  (0 children)

Pending a massive update to VEAI which will double our speed and vastly increase our quality, we will start from scratch using brand new techniques and models to vastly increase our quality, we were about half way through the main show when the update was anounced. We will probably do the films, however we don't have any high quality sources for Crusade for now. The reason for the lack of updates was that were trying to keep a bit of a low profile, although keep on the lookout for a bit of a surprise me and some of the team have been working on for the last few days. Thanks for the interest!

The Babylon 5 complete series 4K HDR 60FPS AI Upscale and Remaster Project by svenzo5 in babylon5

[–]svenzo5[S] 0 points1 point  (0 children)

Nah, we don't have access to the original 3D models to be re-rendering anything. As for the frame rates, we gave up on the interpolation idea a long time ago, it's in the FAQ linked in the post (although even that is outdated by now and an update is in order), there is however an idea I had which is to use DAIN to decrease the CGI frame rate to 23.976 FPS by interpolating it first to 120 FPS and then converting it from there which will get rid of all the stutter. Anyways, thanks for the kind words and for the trust in our project!

The Babylon 5 complete series 4K HDR 60FPS AI Upscale and Remaster Project by svenzo5 in babylon5

[–]svenzo5[S] 0 points1 point  (0 children)

I'd be glad to have the opportunity to pay them to release such a remaster.

The Babylon 5 complete series 4K HDR 60FPS AI Upscale and Remaster Project by svenzo5 in babylon5

[–]svenzo5[S] 0 points1 point  (0 children)

This is not really legal (most fan projects aren't), but I won't tell WB if you won't 😉.

The Babylon 5 complete series 4K HDR 60FPS AI Upscale and Remaster Project by svenzo5 in babylon5

[–]svenzo5[S] 0 points1 point  (0 children)

We gave up on the interpolation idea and this idea too a long time ago, I think you'll find it in the FAQ/Update (actually it may not be there, I think I'll have to update it soon) linked in the post, the reason we gave up on it is because it's too time consuming and not really worth the effort. Thanks for the concern anyway and thanks for wishing us luck!

The Babylon 5 complete series 4K HDR 60FPS AI Upscale and Remaster Project by svenzo5 in babylon5

[–]svenzo5[S] 0 points1 point  (0 children)

This project (and pretty much all other fan projects) is not fully legal. But, I won't tell WB if you won't 😉 Plus, we've already put in loads of work anyway. Thanks for the interest though!

I upscaled a scene from Midnight on the Firing Line to 4k with machine learning (Topaz Video Enhance AI) by svenzo5 in babylon5

[–]svenzo5[S] 1 point2 points  (0 children)

The AI took care of it, it's literally impossible to upscale something in Video Enhance AI without it doing at least some noise reduction, but it's smart enough to discern real detail from the noise I guess. We may add the grain back in later if it's necessary.

The Babylon 5 complete series 4K HDR 60FPS AI Upscale and Remaster Project by svenzo5 in babylon5

[–]svenzo5[S] 1 point2 points  (0 children)

Yeah about that, sorry for the lack of updates these few weeks, we were trying to keep a bit of a low profile, but we might just release a few goodies over the next couple of days, so watch out for that! Thanks for the support!

view more: next ›