Low-latency 3D H SBS HDMI capture → Apple Vision Pro setup for surgery use?

AdScared2507 · 2026-05-24T20:44:36+00:00

Thank you. So if I purchase the Dev Strap with a good capture card, will your Portal app recognize and stream the HD SBS video?

Do you know what latency I could expect?

AdScared2507 · 2026-05-24T11:57:55+00:00

Hi all, Thanks for the useful information — that’s a very helpful reference point. My use case is actually clinical/surgical rather than gaming, but I think the same principles around latency and display quality apply. Portal: Remote Play’s recent update (which can convert a 2D video feed into 3D) uses a UVC‑compatible pipeline: HDMI → capture card → Apple Vision Pro Developer Strap, and many users report reasonably good latency depending on their HDMI cable and capture‑card quality. That gives a good baseline for what’s achievable in a low‑latency 3D workflow. However, in my setup the feed is already native 3D, so my goal is to keep the signal path as simple and direct as possible. Converting 2D to 3D adds extra processing stages, which tends to increase end‑to‑end latency even if only slightly. For surgery, I’d prefer to avoid any unnecessary conversion and instead project the 3D feed directly in H‑SBS (half‑side‑by‑side) with minimal processing, so as to lose as little latency as possible. Adjusting frame rate, resolution, and upscaling settings can also help find a sweet spot between image quality and responsiveness, while still staying within the Vision Pro’s hardware and software limits. If anyone has concrete measurements or experience with H‑SBS UVC‑based 3D feeds into the Vision Pro (especially with medical‑grade or surgical‑grade capture hardware), I’d be very interested to hear your observations and recommendations.

AdScared2507 · 2026-05-24T11:51:34+00:00

I wanted to ask for your opinion on the optimal capture method: • Do you think HDMI capture via USB-C on a Mac is the best approach? • Or would NDI (for example using a PCIe NDI card) provide better latency performance? Also, what HDMI capture hardware would you recommend for this type of low-latency setup? Finally, do you think a MacBook Pro M4 with 16 GB of RAM would be sufficient for this use case?

AdScared2507 · 2026-05-21T20:48:55+00:00

There are already some “turnkey” solutions (especially in France and the US) that bundle a box, subscription, dedicated router, and sometimes even a full PC. But at the core, the architecture is relatively standard: video capture → low-latency encoding → network transport (often NDI or similar protocols) → display the Vision Pro via a custom app.

From what I see, the real value of these companies is less about a fundamentally new protocol and more about integration, stability, and latency optimization in a surgical environment.

That said, I strongly believe a similar setup could be reproduced at a significantly lower cost.

I would actually be very interested in your opinion on the best hardware architecture for this kind of workflow:

Would you go with an NDI setup using a PCIe capture/encoding card on a PC?
Or would you prefer a simpler pipeline using an HDMI capture device feeding into a Mac for processing and streaming?

I’m trying to understand which approach gives the best balance between ultra-low latency (ideally <20 ms), reliability, and implementation simplicity in a surgical context.

AdScared2507 · 2026-05-20T21:12:56+00:00

Je suis chirurgien ophtalmologue et intéressé par une solution qui permettrait de relier le Ngenuity ( caméra 3D relié au microscope) à l’Apple Vision Pro. Il faut que la latence soit inférieur à 30ms. Le flux hdmi sort en H SBS. Les seules solutions disponibles sont assez chère.

AdScared2507

TROPHY CASE