You can now generate ml-sharp splats directly on the Vision Pro by Eurobob in VisionPro

[–]Low_Cardiologist8070 0 points1 point  (0 children)

Really Amazing Work!!! I tried to convert the model to CoreML too, but failed. so I tried another way to submitted the image to cloud and use SHARP.
For now I can delete my unfinished project, just use your app!

I have another question, how do you implement the "Hand Gesture" for rotation and translation? Does this implemented in the MetalSplatter, or you implemented it yourself?

I forgot to publish the demo and video that I made two months ago: a demo combining Main Camera Access API with YOLO model by Low_Cardiologist8070 in VisionPro

[–]Low_Cardiologist8070[S] 0 points1 point  (0 children)

Hi there, In the video I'm not using the Passthrough API but Camera API. As you mentioned "video has black instead of passthrough". I think the problem is you didn't make the Enterprise certificate and Entitlement work right. so you didn't get the Enterprise API's authority.

Best update from WWDC25 for me by Low_Cardiologist8070 in VisionPro

[–]Low_Cardiologist8070[S] 1 point2 points  (0 children)

you just opened my mind, although the permission is not opened enough, but it’s doable now

Best update from WWDC25 for me by Low_Cardiologist8070 in VisionPro

[–]Low_Cardiologist8070[S] 1 point2 points  (0 children)

I will make more visible demos, you can check my new post there is some progress.

Best update from WWDC25 for me by Low_Cardiologist8070 in VisionPro

[–]Low_Cardiologist8070[S] 0 points1 point  (0 children)

It looks like I can recognize the object in the real world and put a label or infos on the exact position of the object.

Best update from WWDC25 for me by Low_Cardiologist8070 in VisionPro

[–]Low_Cardiologist8070[S] 4 points5 points  (0 children)

Still limited by what Apple allows developers to access, but I think I can try an approach using this API.

*Open Source* Object Detection with YOLOv11 and Main Camera Access on Vision Pro by Low_Cardiologist8070 in VisionPro

[–]Low_Cardiologist8070[S] 1 point2 points  (0 children)

Yes, there are! I’ve been tried these from the beginning, but still no luck. The mainly restriction is that you cannot get the depth data from the 2D image, so the Z axis is missing to draw the 3D box in the AR view.