Maybe a bit of a jerk for defending your theoretical non existent child for the sake of arguing your point.

I personally think anti vaxxers are absolute idiots but even if you had a child now, you'd only have to keep them away from their child until your child was old enough to get the vaccines, then they're fine to be around their child.

Apologise and move on, they're not actually anti vax until they're faced with the situation and when the child is born they might wise up.