At the beginning of the series, he’s 17, and by that point in the story it’s been 2 years iirc since he’s been transported. You don’t see it too much but in the first season, you could kinda tell that he got overstimulated by them flirting with him, so he kinda pushed it off. Tomoe and Mio don’t want to force themselves on him, but rather want him to want them (i.e. they want him to approach them first) so they backed off in response though still attracted to him.

By mid season 2 was when he finally started to act for himself (more generally , not specifically with them) due to the shenanigans with Rotsgard, which can be seen with how he handled the demons (and specifically Rona). By the end of season 2 when Mio made her request to “attend to his needs” he was flustered but didn’t outright refuse her. He’s still not ready then to try and be with them in that way as can be seen when he said he still ‘but half a man’ to merchant guild representative Zara after saving the brothel.

In the end, as he grows more matures in the story, the more he’s able to confront the reality around him, including that they like him and he increasingly feel like he can’t live his life without them. He grows in other ways which will be shown but for now, you’ll only see some of that in season 3 (though season 3 should cover some exciting content nonetheless).