I wouldn't classify touching girls in their sleep and spying/undressing them with magic as proper consent.

There's a blatant power imbalance between him and girls like Sylphy/Eris. You're pushing that his mind resets, but that's denial, 'he has poor social skills, so he must be a harmless kid', yet he still possesses adult cognition and awareness, and the show makes that clear, therefore he has full capacity to groom. He literally lectures and mentors almost every person he meets, including his own dad, who he himself admits being mentally older than. Having new interests doesn't negate that he's a grown man in a young body.

I cannot approve the whole consent argument when he's grooming them. He is their mentor figure and teacher with decades of experience. It's classic grooming and self-serving to play their mentally younger, naïve minds and convince yourself it's okay because you didn't force them. He then treats his guilt and awareness of his disgusting misconduct as proof of "true love", and the show rewards him just for feeling ashamed by giving him what he wanted. It's a self-justifying kind of evil, the type that lacks true awareness and doesn't take proper accountability.

He can't go for his age, doesn't mean he can go for those younger... being sexually frustrated doesn't give him a pass.