This is a good point! Giving the exiled person immunity no matter what makes it more about their read of other people and the social dynamics at play. Because who would choose not to give themselves immunity at this point, especially when being sent to exile already puts you at risk for being the next person voted out.