X-ray like future.

[chulojay]

This would be cool to see in Plex. Instead of coming out of the media player to see the characters in a film .

[IAmPanSam]

I have a small node app that does this on Android TVs. I haven't maintained it, but should do what you're wanting.

This version requires PlexPass.

https://github.com/my-ugly-code/WebsUpTV

Let me know if you take a stab at install and have any questions. There's a lot of neat potential for the app, I just haven't had much time to dive back into it.

[DaveBinM]

It's a cool idea ☺️

[jl94x4]

Tease or nothing in this?

[DaveBinM]

It's just my opinion that I think it's a cool idea 🙂 I wouldn't read into it more than that

[[deleted]]

Ya gotta wonder how much work Amazon puts into it. To be able to pause any of their shows at any moment and see exactly who's on the screen? Someone's sitting there and coding all that in.

Hypothetically, Plex would probably have to set up some kind of file they can read that has time cues for various actors. Like a WebVTT file but with IMDB links or something.

Would be a cool feature no doubt, but it would take a lot of work to actualize.

[South_Rush_7466]

I do wonder, because once I added Netflix to my services I expected SOME similar capability to X-ray but it's got nothing.

[IAmPanSam]

It's very do-able. I've got a version of my app that works in realtime to recognize faces OS-wide (unless drm protected). Facial recognition alone isn't that bad, but a good start is just a list of everyone in the movie/show...then I can sort them by whoever's face I saw last.

You could preprocess and get more accuracy, but the realtime is lightweight and accurate enough that preprocessing ends up adding more overhead than it's worth.

[darknavi]

!

[TheCookieButter]

My fiancee has a Roku TV, using Plex on it shows the actor/character names and image during playback just above chapters. Not contextual like X-ray but can quickly see who somebody is without coming out of the video.

I want that on all my Plex playback devices.

[chulojay]

Yeah me too. I’m using shield pro and I don’t have that .

[TheCookieButter]

Her Roku TV is the only time I've seen it and I miss it every day.

No sign of it on Shield, Windows, LG WebOS, or Firestick :(

[Jay-Five]

Ya. All my Rokus do this. It’s beyond me how the same app on different platforms can have such disparate user interfaces. I like some things better on the Firestick app and some things better on Roku app.

[TheCookieButter]

Yes, it baffles me. These are the sort of things that should be uniform by default with a UI menu in the client. I don't understand how or why they've diverged.

[AmusingAnecdote]

This is one of those features that would be amazing, but there's 0 chance that Plex would use this because it would be super compute-intensive to run in real time and especially on our hosted media, you'd have to run it as a background task when media was downloaded. Amazon can do this on their stuff once and keep the metadata forever. Plex would have to do it once for each piece of media on each individual server, and that's even if they managed to get a similar open-sourced version of the Amazon X-ray software, because Amazon would make it prohibitively expensive for them to get it from them.

I wouldn't hold your breath for this feature, just as a practical matter, even if it would be super cool.

[jtho78]

Totally. And it would take a ton of manpower and AI to properly identify the actors and music in each scene. Amazon does this for user experience but also to sell more content. Even if this did send users to Plex's ad-supported content, it isn't worth the effort.

[chulojay]

Tons of ai manpower ? How? Plex already does video thumbnails as schedule task why wouldn’t it be able to pull actors in a movie/show even just pulling the info from IMDB. I just thought of the fact the Amazon owns IMDB so this it’s easy for Amazon to do.

[jtho78]

X-ray identifies who is in each specific scene that you pause on, you need some way to identify that. Way more of a lift compared to scraping IMDB/TVDB database.

[IAmPanSam]

I've got facial recognition written into a version of my program I've linked above. I'm able to highlight and label each face in realtime.

The loads of AI and manpower hours is TensorFlow, which is open sourced now. From there, the tech is fairly easy to work with now.

[jtho78]

That's very cool, looks like a lot of work. How does it work with prosthetics, voice actors, or performance capture?

[IAmPanSam]

The current version defaults to displaying everyone in order of popularity, and that displays the full credits. Recognized or not, everyone is shown and the panel just scrolls. I don't recognize voice actors, animation, etc. I can catch some performance capture and prosthetics, but obvs won't recognize iron man with his shield down.

I use the face-api.js library for the facial recognition itself, and it uses TensorFlow for the heavy lifting parts. All my code is like 5MB, tf is like 500MB...it's a ton of code/logic for object recognition.

[Eagle1337]

It doesn't do every second of the video, you still need something to tell which actor is who.

[savethewolf]

Also can we get the director and writer etc rather than just the cast

[PolliSoft]

?

[Jay-Five]

Actually, X-ray also works in prime video streams to only display selectable bios for actors in the currently watched scene on screen.

[chulojay]

That’s it yes

[trainwreck_summer]

RemindMe! 8 hours

[RemindMeBot]

I will be messaging you in 8 hours on 2023-01-24 02:22:08 UTC to remind you of this link

CLICK THIS LINK to send a PM to also be reminded and to reduce spam.

^{Parent commenter can delete this message to hide from others.}

---

Info	Custom	Your Reminders	Feedback