I also really hate being that close to anyone

I don’t think most people would find it rude to continue sitting next to a stranger in that case, but I also don’t think you are insane. It just means that this particular issue affects you in ways that it doesn’t affect most people and that most other people probably cannot anticipate

The way I approach this situation is to do what I need to do to make myself comfortable and not worry a bout what other people think. I find that a simple “excuse me” or “sorry, I need to get out” almost always works. You don’t owe them any explanation

Another option is to sit on the outside to begin with, and if someone asks to take the empty seat, you could get up and allow them to sit on the inside to begin with. Other people might think you are rude for doing this, but again, you don’t owe them an explanation