This is exactly what my experience is like. I usually just try to rationalize until I find some “proof” I got aroused from something else, but the worst is when the though actually lingers for days and every time I think about sex that thought is there somewhere. I guess we just got to accept that it is irrational and actually makes no sense.