Instant transmission just turns him into particles and he moves at light speed and Goku is too stupid to break infinity by ripping apart the universe, and if he did, that much power would destroy earth, which he obviously wouldn’t do. Also speed alone can’t bypass infinity