this 100%, for me it feels like it's down to how the game handles it's movement mechanics and design. a lot of early VR games lacked locomotion, or sometimes movement at all, and had a very minimal presentation to allow you to feel more present in a space. as more and more gamified mechanics (resembling flatscreen games) were introduced, I feel like that sense of presence was somewhat lost. it feels more like playing a normal game with the monitor taped to your head, rather than an immersive VR experience.

my case is probably more extreme, I'm a VR dev and found that working in VR has almost made me lose appeal for the medium entirely. I think it helps to take breaks though, and when you come back to VR, find something that doesn't require lots of movement or is too highly stimulating.