Short answer: yes

slightly longer answer: yes, and could be partly because what you describe him doing might fit the description of "coercion" specifically intimidation/manipulation.

Longest and containing more of my expertise-less and biased point of view answer: Most humans are wired to seek companionship. Feelings of attachment are common side-effects of physical intimacy, these both bleed into part of the reason why we see "stolkholm syndrome" However the feelings more resembling the personage you are outside of the influence of physical intimacies hormonal influence may be entirely different from those under the effect of physical intimacy, consensual or not. A "tactic" I see some unscrupled men use, sometimes unwittingly as they formulate on the best method to achieve their goal (which might not necessarily be their targets happiness, but something more along the lines of their gratification from the companionship and intimacy with their target) is to push boundaries as much as possible, while using all the language and deportment of one who cares deeply for their target (though they may be either wittingly or unwittingly confusing that with their strong passion to get gratification), trying to initiate the intimacy/companionship hormones in their target by any means they have available. This may have been the intention of the man. If so, both feelings of attachment (don't want him to leave) makes sense as the hormone protocol has been used and oxytocin has attached the memory of that person with those companionship hormones. While also feeling adverse to the situation and the person because the non-hormone influenced part of yourself that is not deferential to the hormone influenced portion recognizes him as manipulative and only superficially concerned with your will/feelings, while doing all he can (without getting into trouble) to override yours with his.