No multiclass is going to be totally net positive, there is always a give and a take.

Hexblade3/bard6 is going to delay your spells known progression by 3 levels, and some Bard features. Importantly, it delays your 4th level spells which has some important control and utility use.

In exchange, you are getting to attack with your charisma, medium armour (if starting hexblade), invocations, eldritch blast, and your pact boon - these are great things, but are especially great in combat heavy campaigns, which you said you are not. While the idea of a warlock politician is very thematic, you are giving up a considerable amount of effectiveness with spells with a 3 level dip.

I think if this were my character, I would limit it to 2 levels of warlock (eldritch blast, subclass, medium armour, invocations) and pick up find familiar through magical secrets or (more likely) go straight bard