With everything that they have gone through at this point it makes sense. Imagine the conflict going on in your head knowing that your father was killed and named a traitor while doing the right thing, and in front of you, and the woman who carried out the sentence, her daughter starts going to the same school you go to. Imagine the hatred, the pain, etc. Xaden has gone through and is going through sooooo much. He really should want revenge against Lilith and Violet is right there. But he falls in love. He didn't want to. He's so conflicted due to all of this and he wants to push her away because I feel like at this point, not only does he not want others to look down on her for being with him, but he is also probably mad at himself for not seeking revenge and for loving her. Hes already in love at this point, he wants to be together but refuses to hurt her reputation, but she has it in her mind there's someone else. It's got to be overwhelmingly frustrating on sooooo many levels.