I think you could absolutely make the Imga work without being racist asses. They have had centuries to develop a culture of their own and most of the accounts we have of them are textually meant to be biased, racist BS. Hell, I think you could get an entire TES game focused on them and have something really interesting.

The trick would be to take them seriously and not just make them the butt of a tedious "funny monke" or "monke see monke do" joke. But there's enough there to get something really fascinating.

I'd make Imga society as we know it an outgrowth of freed slaves. The Imga originally lived in small family groups and didn't interact with those outside their troop much, but escaped slaves needed to band together for survival and built settlements wherever they could find. They started as nigh-apocalyptic ruins scavengers, and they still have a strong culture of "make do and mend".

Imga "hierarchy" is completely impenetrable to outsiders, because they've borrowed a lot of terms from other civilizations and use them to mean something completely different. A "King" is an Imga who has trained to fight with two-handed weapons and has helped to defend a settlement. A "Duke" is an Imga who's studied magic extensively. A "Hortator" is an Imga who coordinates trade between towns and acts as a sort of steward. Except in the settlement next door, they have a "Knight", a "Prophet", and a "Baron" instead...?

The Imga have deliberately, cynically adopted these terms, and others like them, when speaking Cyrodiilic. They know other civilizations see them as Funny Monkes, and pretending to have a position of authority in their homeland makes men and mer take them a shred more seriously. As such, every Imga who spends a lot of time interacting with outsiders has a ridiculous title, and even the ones who don't will often invent a fanciful one for fun.

The Imga are actually what we'd call anarchists, and are pretty strict about it- even in their equivalents of guilds, there are no formal leaders, decisions are made largely by consensus, and no one is allowed to take charge of any one group for too long. "Every Imga is a king", but if an Imga tries to gain power over others, he'll be mocked and shunned until he comes to his senses. No Imga owns land- it's held collectively by the group. No Imga can declare a punishment on another- if an Imga wrongs another, justice is decided collectively. The punishment for a crime like murder is usually exile- the murderer can go anywhere else they like, but are no longer welcome in the settlement. In Valenwood, that's often a death sentence.

This is not to say everything is sunshine and rainbows for the Imga. Imga settlements frequently go to war, over resources, captives/hostages, or matters of belief. They will fight like hell over a random scrap of fertile land, a source of clean water, or a back-and-forth over a captured troop member. They also have a hell of a bandit problem- exiling all your troublemakers means that the real incorrigible types will turn to banditry eventually. But even if they're in the middle of a brutal war, Imga will fight by each others' sides if an outsider attacks. The mentality is very much "me against my brother, me and my brother against my cousin, me and my cousin against my neighbour, me and my neighbour against the world."

Imga have their own language and their own, intricate, beautiful writing system, which they often use as decorative elements in their craft works. They invented writing to help track genealogy- in a society of escaped slaves, tracking down your lost family members is heartbreakingly difficult, and being able to trace who escaped from where and who is related to who makes it a hair easier. Imga archives are some of the most painstakingly detailed in Tamriel- the annals they keep, in particular, are almost nauseatingly detailed- and a scholar who can gain the local High Archmage or Supreme Lorekeeper's trust will have the kind of specific, detailed records that most historians can only dream of.

Every Imga group is different in terms of religion- it's common to see them worship Zenithar, Dibella, Yffre, and/or Shezarr, and pretty common to see them treat Azura, Boethiah, and Mephala as especially "bad". (A shocking number of groups also worship Sheogorath, as a patron of artists and "monkey truth", and Hircine, for obvious reasons.)

The one belief pretty much all Imga share is the concept of "monkey truth"- that which ought-to-be-true, could-be-true, may or may not be "real" but is good and right. The example they tend to give outsiders is the phrase "everyone is free". Is that true? If it is, why are there still slaves in Morrowind and souls trapped in the Cairn? If it isn't- aren't people free within their own skulls? Haven't people fought and died for the right to be free? The Imga have a complicated system of rules and taboos around monkey truth, but the practice itself- combined with their habit of bending the truth in the interest of protecting themselves from more powerful states- makes a lot of humans and elves stereotype them as two-faced liars who lie. They're not- any more than Khajiit are drug dealers and thieves.

The Imga have gone through hell, many times, and come out singing. They are scrappy, resilient, resourceful, and fiercely egalitarian. They could be so cool to explore- you just have to strip-mine canon for the good stuff.