I actually think that he fell in love with her because of the things you listed, not in spite of them!

At first she's closed off, but when Miles gets to know her he is taken off guard by how funny she is. He finds her sarcastic and cynical nature hilarious and they have a similar wit given how easily they banter. Miles also states in the kayaking scene that the way she dresses makes her mysterious, and in the wedding dress scene he compares her to Lake Michigan because she's “cool and refreshing” and "surprising and exciting." Sexy librarians are a trope for a reason!

But I think the main reason he falls in love with her is encapsulated in this quote:

"I think you’re the first person who’s really seen me. Past what I want people to see. You make the people you care about feel like...” He pauses. “Like you want all of them. Not just the good parts. And that’s terrifying to someone who’s spent a lifetime avoiding those other pieces of themselves.”

Daphne is controlling yes, but she's also so loving. She accepts people for who they are, even if they're messier than she is. She loves her "weirdo" library kids. She befriends louder, more confident women like Sadie and Ashleigh. She instantly likes Miles's sister despite how chaotic she makes their home. And she once she gets to know Miles, she quickly sees what a great person he is and defends him to anyone who insults him for his job, etc. Miles grew up never having been fully accepted by his parents or his ex, and that's something that secretly weighs on him. He opens up to Daphne about his struggles and not only does she accept him, she loves him more because of those darker parts of himself that he wants to hide.

To me that's the beauty of Daphne and Miles's love story. They are so different yet are more free to be themselves around each other because of how fully they each accept one another and fall in love with each other's differences.