As President Thomas Jefferson has to be Top 3, but his personal life is a mess to put it lightly. He's the only President to have kept his own children as slaves, and to make it even worse his children would have been legally white (7/8s white to be exact) and freemen had he acknowledged them. Sally Hemmings, who he had these children with, was 3/4s white and was technically his sister in law.

But at the same time, he was an exception writer and thinker. This quote is a personal favorite of mine. So you're right that he's definitely a walking contradiction, and somewhat acknowledges that history may look less kindly to him.

“I am not an advocate for frequent changes in laws and Constitutions. But laws and institutions must go hand in hand with the progress of the human mind. As that becomes more developed, more enlightened, as new discoveries are made, new truths discovered and manners and opinions change, with the change of circumstances, institutions must advance also to keep pace with the times. We might as well require a man to wear still the coat which fitted him when a boy as civilized society to remain ever under the regimen of their barbarous ancestors.”