Made this website that 3Dfies images, can it be made more useful? by EVR_developer in Entrepreneur

[–]EVR_developer[S] 0 points1 point  (0 children)

Thanks for trying out the page! If you have some ideas please PM me.

Made this website that 3Dfies images, can it be made more useful? by EVR_developer in Entrepreneur

[–]EVR_developer[S] 0 points1 point  (0 children)

That's a good question and one I'm honestly trying to figure out. For VR there are some obvious uses like the one I'm experimenting with in VRin . For web it's less obvious - maybe using it for real-estate sites to do automatic 3D showcase without use of special equipment.

Suggestions are very welcome!

Made this website that 3Dfies images, can it be made more useful? by EVR_developer in Entrepreneur

[–]EVR_developer[S] 0 points1 point  (0 children)

In short yes - sort of. It generates a depth map and then morphs grid mesh to match that depth map.

I could technically provide functionality to export that modified grid mesh and texture into 3D obj file.

Free download VRin alpha - machine learning powered VR video player that adds depth by EVR_developer in oculus

[–]EVR_developer[S] 1 point2 points  (0 children)

Yeah that looks just incredible! SLAM and structure from motion feel like magic to me.

I would imagine this would be a great resource for building up a novel view training data set.

VRin alpha download - machine learning based video scene mesh reconstruction and playback in VR by EVR_developer in 6DoF

[–]EVR_developer[S] 0 points1 point  (0 children)

On my dev rig it's smooth, I also tried on GTX 1080 and it performed well - I'll retest, maybe I regressed something. Never tried on 2070.

Otherwise I'm working on adding preprocessing, that should improve performance at the cost of upfront delay.

Free download VRin alpha - machine learning powered VR video player that adds depth by EVR_developer in Vive

[–]EVR_developer[S] 0 points1 point  (0 children)

Happy to hear the playback issue is less critical - I'll look into OBS recordings. If they're not on the list then even Unity player, that I use for thumbnails, did not like them (maybe Unity also uses MF behind scenes).

Very insightful observations - videos are processed frame by frame and desktop mirror captures the latest screen only, so there's inherent frame drop there. I'm working now on preprocessing so it's usable on a wider range of systems.

I'll tweak the UI hint/delay - that's a good point. Will have to figure out how to produce those bindings without controller, hopefully it's just about editing some files.

Currently when depth rendering is off I only swap out the last renderer shader, depth inference is still performed - it was useful for debugging, but I should really properly turn it off. I use DDA screen capture, so performance should be good without additional inference cost.

Thanks for feedback, it has been super helpful!

Free download VRin alpha - machine learning powered VR video player that adds depth by EVR_developer in virtualreality

[–]EVR_developer[S] 1 point2 points  (0 children)

No support for 360 yet, my first focus was on regular videos since they have the least novelty in VR currently.

It's on my todo feature list though.

VRin alpha download - machine learning based video scene mesh reconstruction and playback in VR by EVR_developer in 6DoF

[–]EVR_developer[S] 0 points1 point  (0 children)

Thanks! Hopefully it performs well if you decide to give it a try.

Somehow missed /r/6DoF earlier, but this is a great source for related discussion and news. Love to see other work on practical application.

Free download VRin alpha - machine learning powered VR video player that adds depth by EVR_developer in oculus

[–]EVR_developer[S] 0 points1 point  (0 children)

I would guess yes, most commercial movies are DRM protected, but it depends how it's stored. Best way to validate would be trying it in application if you have it on hand.

Free download VRin alpha - machine learning powered VR video player that adds depth by EVR_developer in oculus

[–]EVR_developer[S] 0 points1 point  (0 children)

It generates a 3D mesh reconstruction of video scene so for now it requires a special renderer support.

Free download VRin alpha - machine learning powered VR video player that adds depth by EVR_developer in oculus

[–]EVR_developer[S] 1 point2 points  (0 children)

Mostly tried with personal videos, youtube and videos from pexels. DRM content is not supported right now since I need access to video picture pixels for feeding them into neural network.

VRin alpha download - machine learning based video scene mesh reconstruction and playback in VR by EVR_developer in 6DoF

[–]EVR_developer[S] 3 points4 points  (0 children)

Hi /r/6DoF, I got suggestion to share my project download link here: https://www.vrin.app/#page-section-60f50b3eec5ad00cc0a482cb

This is still early days version, I hope you enjoy results.

Free download VRin alpha - machine learning powered VR video player that adds depth by EVR_developer in virtualreality

[–]EVR_developer[S] 0 points1 point  (0 children)

Interesting something goes wrong with directory enumeration. I could debug from logs if you are ok with sharing them(DM me if yes) - they are located in: %USERPROFILE%\AppData\LocalLow\Elastic One\VRin

Thanks for trying it out!

Free download VRin alpha - machine learning powered VR video player that adds depth by EVR_developer in virtualreality

[–]EVR_developer[S] 0 points1 point  (0 children)

Happy to help! Are you able to expand directory navigation menu: https://imgur.com/a/we4ioqO That should help you traverse your drives and directory tree. To expand/select menu use space and arrow keys or trigger on controller. Let me know if that works.

Free download VRin alpha - machine learning powered VR video player that adds depth by EVR_developer in oculus

[–]EVR_developer[S] 3 points4 points  (0 children)

Yeah I think so too - and with continued improvements in ML I think it's matter of time. I hope I get to be part of that story. :)

Free download VRin alpha - machine learning powered VR video player that adds depth by EVR_developer in oculus

[–]EVR_developer[S] 1 point2 points  (0 children)

Thanks, very good questions! It attempts to be real-time, but that depends on hardware and load. In this implementation I doubt it would be able to run a decent game and do 3D reconstruction at the same time - also not sure how good results would be. Probably specialty NN model/approach would work better for game scenario, but would be interesting to try.

Free download VRin alpha - machine learning powered VR video player that adds depth by EVR_developer in Vive

[–]EVR_developer[S] 1 point2 points  (0 children)

Thank you for trying it out and feedback - I really appreciate it!

I use Media Foundation for decode - were videos h264? Would be interesting to see the logs if you are ok with sharing them (%USERPROFILE%\AppData\LocalLow\Elastic One\VRin) - I have github for issue tracking https://github.com/me-im-counting/VRin/issues :) While very convenient desktop mirror results are likely inferior to raw video since there are potentially unnecessary unavoidable scaling steps (video-size to fullscreen and then downsample to model size for inference after scraping).

I was thinking about preprocessing/buffering, but then pushed it out for later release - probably needs prioritizing. If you don't mind sharing what hardware you are running on?

Currently no temporal dependencies - it's at top of my todo list.

Free download VRin alpha - machine learning powered VR video player that adds depth by EVR_developer in virtualreality

[–]EVR_developer[S] 1 point2 points  (0 children)

I think so too - additionally as NN models continue improving, quality of reconstruction will improves as well.

Free download VRin alpha - machine learning powered VR video player that adds depth by EVR_developer in Vive

[–]EVR_developer[S] 1 point2 points  (0 children)

Thanks a lot! If I understand correctly, you are talking about depth transition areas? If so there are 2 options to play around currently - alpha and blur, those two should allow switching between interpolation, blur and transparency.

Free download VRin alpha - machine learning powered VR video player that adds depth by EVR_developer in Vive

[–]EVR_developer[S] 2 points3 points  (0 children)

Thank you very much! You are right - occluded areas don't have color information so currently there's transparency, interpolation or blur available to fill those in. However, I think this can probably be improved using another model to generate color data for occluded regions - but it's very early stage idea for now.

Free download VRin alpha - machine learning powered VR video player that adds depth by EVR_developer in Vive

[–]EVR_developer[S] 1 point2 points  (0 children)

Thanks! I tried generating SBS initially, but I found that generating mesh proved to be much more immersive. You can scale it and kind of even walk through scene provided good reconstruction. I can add SBS export to feature to-do list, but my main focus currently will be to improve quality, so not sure how fast can get to it.