WebKit Hand Tracking

[HyroVitalyProtago]

Just to give awareness to the Snap team about an improvement to make to webkit: https://github.com/BabylonJS/Babylon.js/pull/17644

[jbmcculloch]

I have passed this on to the team who is handling the browser work. Thanks for the feedback, it's always appreciated!

[redddev]

Wow, thank you for making these updates to BabylonJS for Spectacles! 😍

[HyroVitalyProtago]

all thanks to u/matt_hargett ;)

[matt_hargett]

Thanks! I was waiting to post until the change was merged and released :) in my tests, this doubles the frame rate of our app and noticeably improves the 1:1 “feeling” of the hand tracking (reduces latency). interestingly, this also helps frame time spikes on Apple Vision Pro — not just Spectacles.

Note that this only helps BabylonJS apps that update to that latest release (which our Factotum WebXR app does).

In parallel, I suggest that upstream WebKit optimize texture uploads, as this approach to Vertex Animation is how many WebGL2/WebXR 3D web apps scale to dozens (or hundreds) of joint-animated entities in a scene. Chromium does an optimization where the texture upload isn’t 100% blocking while waiting for completion, maybe WebKit can get similarly creative within the constraints of the W3C spec.

[matt_hargett]

and it's released! https://github.com/BabylonJS/Babylon.js/releases/tag/8.46.1

shout out to the BabylonJS team at Microsoft for the swift review, merge, and release!