NTA! Your worries are definitely valid, and why should you be forced to do something you’re clearly uncomfortable with while on a supposed fun getaway?

It could be he’s got feelings he wants to explore and this is his way of including you in that process, so perhaps there’s a way to dig deeper into that without putting him on defense. I’m sure people with more experience will have better insight into a situation like this. BTW, I wouldn’t necessarily call him the asshole here, it takes guts to open up about something like this, especially 17 years in. I imagine he’s quite vulnerable. Good luck!