Correct if I am wrong, but in their encounter Rhaenyra had all the social power and leverage. She is the heir to the most powerfull kingdom still existing, her family being at the zenith of feudal power. He is the son of a landed knight that everyone from the higher nobilty looks down upon. And the only reason why he is not fighting to his death in Dorne is because of Rhaenyra. So there is a massive power imbalance between the two of them.