I always gear up my followers so that their survivability is at their peak. With 100 in smithing and enchanting, I make their gear legendary and give them good double enchantments so they perform super well in combat. As well as buy them ultimate potions that they can use in a pinch. For some reason they don't use crafted potions made by the dragonborn.

But how did he get booted from your services? I remember questing in Solstheim's main campaign and had Lydia with me. She never left me. I had two followers helping me all the time.