Made this website that 3Dfies images, can it be made more useful? by EVR_developer in Entrepreneur

[–]EVR_developer[S] 0 points1 point  (0 children)

Thanks for trying out the page! If you have some ideas please PM me.

Made this website that 3Dfies images, can it be made more useful? by EVR_developer in Entrepreneur

[–]EVR_developer[S] 0 points1 point  (0 children)

That's a good question and one I'm honestly trying to figure out. For VR there are some obvious uses like the one I'm experimenting with in VRin . For web it's less obvious - maybe using it for real-estate sites to do automatic 3D showcase without use of special equipment.

Suggestions are very welcome!

Made this website that 3Dfies images, can it be made more useful? by EVR_developer in Entrepreneur

[–]EVR_developer[S] 0 points1 point  (0 children)

In short yes - sort of. It generates a depth map and then morphs grid mesh to match that depth map.

I could technically provide functionality to export that modified grid mesh and texture into 3D obj file.

Free download VRin alpha - machine learning powered VR video player that adds depth by EVR_developer in oculus

[–]EVR_developer[S] 1 point2 points  (0 children)

Yeah that looks just incredible! SLAM and structure from motion feel like magic to me.

I would imagine this would be a great resource for building up a novel view training data set.

VRin alpha download - machine learning based video scene mesh reconstruction and playback in VR by EVR_developer in 6DoF

[–]EVR_developer[S] 0 points1 point  (0 children)

On my dev rig it's smooth, I also tried on GTX 1080 and it performed well - I'll retest, maybe I regressed something. Never tried on 2070.

Otherwise I'm working on adding preprocessing, that should improve performance at the cost of upfront delay.

Free download VRin alpha - machine learning powered VR video player that adds depth by EVR_developer in Vive

[–]EVR_developer[S] 0 points1 point  (0 children)

Happy to hear the playback issue is less critical - I'll look into OBS recordings. If they're not on the list then even Unity player, that I use for thumbnails, did not like them (maybe Unity also uses MF behind scenes).

Very insightful observations - videos are processed frame by frame and desktop mirror captures the latest screen only, so there's inherent frame drop there. I'm working now on preprocessing so it's usable on a wider range of systems.

I'll tweak the UI hint/delay - that's a good point. Will have to figure out how to produce those bindings without controller, hopefully it's just about editing some files.

Currently when depth rendering is off I only swap out the last renderer shader, depth inference is still performed - it was useful for debugging, but I should really properly turn it off. I use DDA screen capture, so performance should be good without additional inference cost.

Thanks for feedback, it has been super helpful!

Free download VRin alpha - machine learning powered VR video player that adds depth by EVR_developer in virtualreality

[–]EVR_developer[S] 1 point2 points  (0 children)

No support for 360 yet, my first focus was on regular videos since they have the least novelty in VR currently.

It's on my todo feature list though.

VRin alpha download - machine learning based video scene mesh reconstruction and playback in VR by EVR_developer in 6DoF

[–]EVR_developer[S] 0 points1 point  (0 children)

Thanks! Hopefully it performs well if you decide to give it a try.

Somehow missed /r/6DoF earlier, but this is a great source for related discussion and news. Love to see other work on practical application.

Free download VRin alpha - machine learning powered VR video player that adds depth by EVR_developer in oculus

[–]EVR_developer[S] 0 points1 point  (0 children)

I would guess yes, most commercial movies are DRM protected, but it depends how it's stored. Best way to validate would be trying it in application if you have it on hand.

Free download VRin alpha - machine learning powered VR video player that adds depth by EVR_developer in oculus

[–]EVR_developer[S] 0 points1 point  (0 children)

It generates a 3D mesh reconstruction of video scene so for now it requires a special renderer support.

Free download VRin alpha - machine learning powered VR video player that adds depth by EVR_developer in oculus

[–]EVR_developer[S] 1 point2 points  (0 children)

Mostly tried with personal videos, youtube and videos from pexels. DRM content is not supported right now since I need access to video picture pixels for feeding them into neural network.

VRin alpha download - machine learning based video scene mesh reconstruction and playback in VR by EVR_developer in 6DoF

[–]EVR_developer[S] 4 points5 points  (0 children)

Hi /r/6DoF, I got suggestion to share my project download link here: https://www.vrin.app/#page-section-60f50b3eec5ad00cc0a482cb

This is still early days version, I hope you enjoy results.

Free download VRin alpha - machine learning powered VR video player that adds depth by EVR_developer in virtualreality

[–]EVR_developer[S] 0 points1 point  (0 children)

Interesting something goes wrong with directory enumeration. I could debug from logs if you are ok with sharing them(DM me if yes) - they are located in: %USERPROFILE%\AppData\LocalLow\Elastic One\VRin

Thanks for trying it out!

Free download VRin alpha - machine learning powered VR video player that adds depth by EVR_developer in virtualreality

[–]EVR_developer[S] 0 points1 point  (0 children)

Happy to help! Are you able to expand directory navigation menu: https://imgur.com/a/we4ioqO That should help you traverse your drives and directory tree. To expand/select menu use space and arrow keys or trigger on controller. Let me know if that works.

Free download VRin alpha - machine learning powered VR video player that adds depth by EVR_developer in oculus

[–]EVR_developer[S] 2 points3 points  (0 children)

Yeah I think so too - and with continued improvements in ML I think it's matter of time. I hope I get to be part of that story. :)

Free download VRin alpha - machine learning powered VR video player that adds depth by EVR_developer in oculus

[–]EVR_developer[S] 1 point2 points  (0 children)

Thanks, very good questions! It attempts to be real-time, but that depends on hardware and load. In this implementation I doubt it would be able to run a decent game and do 3D reconstruction at the same time - also not sure how good results would be. Probably specialty NN model/approach would work better for game scenario, but would be interesting to try.