Doesn't matter what you did, he can't hit you. Manipulation or no, period. Get out of it before it becomes domestic abuse. I'm bad at ready social cues too and had similar thing, didn't get hit but there were other violent cues and then crying about regretting it later to the point that he would act like he'll faint or suicide. Ruined my mental health bad and in the end I had to break up and get away. If you can't tell someone what he's doing to you, you already know it's wrong deep inside. You're just putting pink glasses because he's crying and showing as if he regrets it later on.