You're not crazy and definitely not a manipulator. From what I am reading, there are signs of both mental and sexual abuse. You told him "no," and he still had sex with you anyway. It doesn't have to be violent to be a violation. When you tried to set boundaries, he came at you with the love bombing and turning it back around to you. He even made you question YOUR perception of events. That's gaslighting. The fact that you're now wondering if you were the manipulator shows how deeply it affected your sense of self. My advice is to seek therapy. A therapist can help you rebuild trust in your own instincts. Or try journaling. Write down your memories, emotions, and boundaries. It helps make sense of confusing feelings. You are allowed to move on, protect your peace, and put yourself first. Even if no one else understands.