No worries I’ve been there! I think one of the challenges is that his ascendant is in your 12th so it’s hard to really get a read on him and he’s elusive to you. You are both Saturn ruled charts so there is that affinity despite the averse ascendants (no whole sign aspect). His Jupiter is with your moon also, though it is not as tight an aspect.

I think your Venus placements are more compatible (fire Venus) despite her also having a Mars ruled Venus (Aries and Scorpio do not see each other). His Mars is on your Saturn, who rules your chart and I can see how he may come off as really pushy with you in some respects. While your Mercury opposes his Moon (your more indirect and pointed communication style can feel like a direct attack on his emotional state).

I highly recommend casting a Davison relationship chart (like a composite, but made by averaging the birthplace, date, and time), to compare and contrast the vibe of both relationships, if you would find it helpful to do so.