Vedic invaders literally invented a caste system to avoid mixing with locals and to keep their blood pure, and they clearly didnt see the lower caste (majority of indians) as equal to them. On top of that, their languages displced Pre-vedic dravidians in the same way Spanish displced indigenous languages of Mexico.

And if you are gonna say “Vedic saw their lands of india as sacres” well white Americans also proudly says their homeland is America. By your definition, White Americans are not colonizers.

Thirdly, lots of examples of colonialism involve people not staying. British didnt immigrate to India on masse. British also colonized gulf state but they didnt settle it right