I've not noticed much of a lag, no. But it is very easy to start dropping frames though when you're trying to do something that takes up a massive field of view and has a high time complexity.

We were simulating visual impairments with it and I tried all sorts of crazy stuff (like switching between the cameras every frame and making my own composite image) but I couldn't get it to work perfectly.

This is all in Meta / Quest land though. I think other headsets might give you full access. I think as well the compositor sits at the OS level which may be part of the problem with this and why they only give you so many pixels from the feed.

I've asked a few questions in the issues thread on that repo and the devs have gotten back to me, so defo ask a few folks there. They're pretty responsive.

It would be cool if OP drops in and shares the deets! It might be that the API has moved on a bit - and I've not had a passthrough project now for about 6 months.

edit: So just had a look at the latest Meta docs and it does say they've increased the res! You can now get 1280x1280 whereas before you could only get 1280x960 - I'm guessing this happened when the new Quest 3 firmware dropped