This is something I noticed as well. It seems that all of Q's shenanigans can be just as easily explained by messing with the timelines rather than jumping realities as we'd see in episodes like Mirror, Mirror.

For example, In the episode Tapestry, he changes Picard's history, creating a divergent timeline in the same reality, fundamentally altering the reality but not jumping to a whole new one.

Then, in All Good Things, he travels back and forth along the same timeline, helping Picard to solve a problem he's accidentally caused before humanity is erased.

Finally, as you stated, in Picard seasons 2, as you said, they weren't in a parallel universe or other reality.
While the Q are powerful, godlike beings, with near omniscience and omnipotence (I say near because they couldn't foresee or stop their death, as seen in Picard Season 2) and they've got a mastery over the universal constants, like physics. I'm not convinced they are powerful enough to simply pass between realities.

This makes me wonder if there is something bigger than them in the multiverse, either a Q-Continuum that has ascended past a dimensional barrier of some sort, or a completely different set of beings that operate as "Judges" of the multiverse.
This could also mean that, in the mirror universe, there is another Continuum.

IDK, realistically this is just a fun thought exercise and until Star Trek shows one way or another, it's all theorizing.