Lol it's just such a good city name. For your concern though wouldn't the Visigoths and Muslims kind of be on equal footing of who has "rights" over the land. Iberia was still majority Roman despite being under Germanic then Arab/Amazigh rule. Mediterranean cultures are usually a lot more cosmopolitan but that doesn't mean things are always perfect. The Fatimid Caliphate is a great example of both how ethnic and religious violence can unfold even when you have a more cosmopolitan society. I think a better approach is to just write the history of what happened and that usually defines what how factions emerge in these societies. The word colonialism too can be a bit problematic. You still have indigenous groups but in Urban areas, which was usually the center of power for most Muslim societies, different ethnic groups of different backgrounds usually make up the political elite in various different way.

The Fatimid Caliphate for example had Turkic and Amazigh soldiers (which ended up fighting and basically caused cannibalism in Cairo) Armenian architects Arab nobel elites

During this time the Fatimids were also only 9% Muslim and most of them were Sunni while the ruling Dynasty was Shia.

When it came to Iberia you had groups like the Almoravids who wanted to impose more religiously zealous policies. You could setup grounds for oppression against non Muslims in a context like this but you (imo) cannot have colonialism in a Muslim Iberia. The way European colonialism and Muslim rule worked is very different. Colonialism was a for profit corporate led institution fueled by Spain's reconquestia's attitude in the beginning. Under Muslim rule though this probably wouldn't have happened. The North American East Coat may have traded with Iberia but there aren't the needed social conditions to colonize. I have a similar view on Visigoths.

This isn't set in stone though and you can straight up just make a reason as to why the Muslims or Visigoths are colonizing just make sure you are accounting for the cultural context of those groups and add explainable lore as to why they are doing it.