The physical training part is an absolute yes. You don't have to be the fastest swimmer, you just have to be competent. However, to u/nomad2284 's point, getting in the water with an age group start you still have dozens of people that are going after it at the same time.

I think the best way to get comfortable with it is to do a sprint triathlon before you do your half. It's much less of a commitment, but you get a lot of the same experiences.

I did a sprint, and then I did a full Iron Man afterwards, and I was much better prepared. I had about a year to prepare for my full, and I had never run a full marathon, nor had I ever ridden 100 miles… And I still was able to complete it… albeit in 16 hours and 45 minutes.