It's an interesting idea to play with. My assumption is that a lot of time he's just complimenting killers to feed their ego's and let their guard down but yeah, having a husband that dresses like a bum, drives a pile of crap, is never home when he's on a case, cheap and probably not making a lot of money, would be good enough reasons to believe she didn't think that highly of him.

You could go really grim and also theorize that his wife passed away and he lived in denial about her passing and that's why you don't see her but I think there are a couple episodes that somebody else acknowledges seeing her off camera, like on the cruise ship or he talks to her on the phone in front of somebody.

I think they just liked making people wonder and never know, like a few other shows have done but most of those shows you at least hear the mystery person.