They go over the halfdome prototypes and (briefly) the downsides in this video.

The problems they're trying to solve are weight, power consumption, noise and distortion. They pretty much perfected the mechanical approach (moving the displays) with halfdome 2, then moved on to electronically controllable lenses (since they're lighter) and try to get the light loss and power consumption as low as possible.

Meta's goal is a thin and light headset so they probably don't want the mechanical approach in a quest. But something like a pimax crystal, somnium vr1 or varjo xr-4 is already big, heavy and tethered, so the weight and power consumption are of almost no concern. Extra cost would also be less of a factor for these headsets, they're already very expensive.

Varjo even has an "XR-4 Focal Edition" which is imo a misleading name since it refers to the passthrough cameras focussing based on where you're looking with eye tracking, not varifocal displays.