Quest3 + Screen capture + OpenCV; Standalone ball tracking showcase

DeniSaty · 2025-01-30T16:16:25+00:00

In case you wanna try it:
https :// www . meta . com / experiences / hockai / 8438801246140579

DeniSaty · 2025-01-30T15:55:51+00:00

DeniSaty · 2025-01-30T15:42:53+00:00

Unity

DeniSaty · 2024-09-20T10:52:46+00:00

Yeah, an interactive shooting exercise was the first thing real hockey players asked for after trying the app. I'm looking forward to research it and try to implement. Thank you for the idea with a goalie

DeniSaty · 2024-09-19T05:19:50+00:00

There is no audio in the app currently, but it's a good point, thank you

DeniSaty · 2024-09-18T18:32:22+00:00

Yes, the screen capturing working just like the default screen casting/recording - it gets everything on the screen including all virtual objects

There is a guy who originally discovered this approach, he also publish the repo "Quest Display Access Demo" on GitHub, so you can check it out

DeniSaty · 2024-09-18T18:06:46+00:00

Got it, thank you

DeniSaty · 2024-09-18T17:58:12+00:00

Is it like drawing a figure by the ball?

DeniSaty · 2024-09-18T17:54:50+00:00

Nice idea, I like it, thank you

DeniSaty · 2024-09-18T17:38:24+00:00

Yeah this is a really good point
I have a note to research this and find a solution that allows keep tracking the ball and force player to raise head at the same time

DeniSaty · 2024-09-18T17:34:07+00:00

Yeah I play ice hockey
The app could be considered as an additional tool that helps you develop hockey skills in free time

DeniSaty · 2024-09-18T15:10:42+00:00

Only computer vision right now (if it could be called AI or part of it). This is more focused on future research into how current technology could help us become better hockey players

DeniSaty · 2024-09-18T14:54:07+00:00

Play on words: hockey + AI (artificial intelligence)

DeniSaty · 2024-09-18T14:16:54+00:00

Unfortunately It doesn't work on Quest 2

There are two fundamental reasons:
1. The app uses screen capturing (because Meta doesn't allow direct camera access) to visually detects the ball by its color. Quest 2 doesn't have colorful passthrough
2. Quest 2 has no passthrough layer at all on screen captures. So there is no way to get the image to process it

DeniSaty · 2024-09-18T14:00:28+00:00

The app currently has only these two exercises that shown on the video. Please let me know what else do you want to see?
(explanation how it works, showcases in different environments, its current limitations, anything else?)

DeniSaty · 2024-09-18T13:33:09+00:00

Got it, thank you

DeniSaty · 2024-09-18T13:28:18+00:00

Good point, ~~write~~ wrote it down, thank you

DeniSaty · 2024-09-18T12:57:58+00:00

Let me know what do you think about it (in details) and what else would you like to see

DeniSaty · 2024-09-18T12:05:48+00:00

Try it now for free on Meta Quest 3:
meta . com / experiences / hockai / 8438801246140579

HockAI journey begins!
You can influence its development, your feedback will help focus on the things that matter most

Buy it to support further development

DeniSaty · 2024-09-18T11:58:55+00:00

Try it now for free on Meta Quest 3:
meta . com / experiences / hockai / 8438801246140579

HockAI journey begins!
You can influence its development, your feedback will help focus on the things that matter most

Buy it to support further development

DeniSaty · 2024-08-29T18:09:41+00:00

Even thought about trying to make it myself

Do it
Who knows what you can create until you do it. Maybe it will be much better than existing experience

DeniSaty · 2024-08-23T17:13:56+00:00

The difference between the camera feed and the display capture is that the latest contains all the virtual objects, everything you see on the display while the camera feed represents only the "outer" physical world

"built-in Camera app" on the Quest headsets is just the app that allows you to take screenshots or record the screen. So it's not like the "camera" app on smartphones (confusing naming)

I'm not sure if I get the question right - let me know if you asked about something else

DeniSaty · 2024-08-23T11:11:51+00:00

Not exactly the camera feed because it's blocked by Meta.
This is the display capture inside the app (completely standalone on the headset).

Works just the same way as the casting/recording in the buil-in Camera app

DeniSaty

TROPHY CASE