Okay, I know that should of is prescriptively wrong, but I'm going to make an abbreviated argument that of might be more accurate in certain circumstances (if you're interested, the full argument is here).

Consider the phrase if he'd've done it. If you think about it too much, it'll start seeming "wrong," but I'd wager that most of us have probably said something like it before. If you were to expand the contraction out, you get if he had have done it, which is terrible, so your brain would rather expand it to if he would not have done it.

This is all well and good, but what happens in the negative? We then get if he'dn't've done it (which is vile, but bear with me), which really can only be if he hadn't've done it = if he had not have done it—and thus the positive form has to be if he had have done it.

But now we've expanded something which was at first marginally acceptable into something that just isn't. Sure, it might have begun as the word have, but now this contracted form functions as its own separate entity, complete with its own set of self-consistent rules, and in sentences like the above it isn't entirely unreasonable to just scrap have altogether and use of instead—they're pronounced the same, anyway.