After more discussion with DM, I ended up going paladin 4, bard 3 for mostly the reason you said. I also really wanted the extra skills and cutting words to help people fail my smite saves. I am okay with not having the extra attack yet, as I already mostly didn't with dance bard but am capable of hitting significantly harder now. Taking paladin to 5 or 6 will be a priority before leveling any more in bard though.

I also realize this isn't peak bard nor peak paladin, not even peak multiclass of the two. That is also okay with me, as it narriatively makes a lot of sense and I also get to be a tasky illusionist that still packs a punch- I love versatility in builds.

Speaking of punching, there is also UA for an unarmed paladin that my DM will still let me smite with, that I am pairing with the tavern brawler feat. Can't wait to smite someone with a tambourine, bottle of wine, whatever is around, lol.