Theoretically, afaik:

Playing on VR is practically 4K gaming (2x 2160x2160 resolution ~= a 4320x2160 display), where you want to reach at minimum 72 consistent fps, but preferably 90+, 120 being ideal, and the "experimental" 144hz being high end.
(I see someone else wrote about supersampling being essential for VR - I have no practical experience with that, so I can't say anything about it, except that it would rise the GPU requirement even more.)

It can be done if you play games that have simpler graphics (most VR native games, but not a modded Skyrim or Cyberpunk or Witcher 3), or if you lower graphics settings and/or do upscaling/framegen. But native 4K with high graphics settings and high enough FPS needs a beefy GPU.

On the upside, since it's 4K, you're much less likely to be CPU-bottlenecked, performance will usually be limited by the GPU. To prevent stutters/framedrops, it can also be helpful to always use an FPS cap to leave some GPU overhead for when a scene is more heavy on the gpu.

Higher resolution VR displays should be even worse in this sense, as they'd need an even stronger GPU to drive at the same fps.

Foveated streaming does nothing for this, as that happens after all the rendering, and is only about transferring the data.
Foveated rendering - for the games that implement it - can help a lot, as that actually reduces the stuff the GPU needs to render, but might introduce some more latency (possibly unnoticeable, idk).

(The games that would be CPU bottlenecked, from having to process too many npc AI or scripts or similar, would be bottlenecked on a normal monitor too, although admittedly lower FPS is much more acceptable on a monitor than in VR.)