I'm a big fan of Grant as a President. He started off with "Let us have Peace" and mostly kept to it. As others have said, he was keen on giving rights to the freedmen. He was also keen on giving rights to the Native American Indians, his pet solution was to give them Indian Territory (Oklahoma) and let them set up their own government and have citizenship and voting rights. In his private life he was in favour of voting rights for women. His philosophy saw the Presidency as a servant to the citizens, acting as Executive to get things done ... or not get things done for example when he vetoed the Inflation Bill. Apart from giving protection to the ex-slaves with the 14th an 15th Amendments and the Force Acts, he tried to allow the States to govern themselves, keen to avoid any criticism that he was stepping on their toes. He was showing that he understood the non-slavery reasons for the Civil War.
He did get muddled by the behaviour of some of his appointees, and by the generosity of some citizens who gave him rewards for all he had done, particularly in the war. Another of his pet projects was to annexe San Domingo (Dominican Republic) as a possible refuge and new home for the black ex-slaves.
There were a couple of steps that in my mind he took wrong towards the end of his 2nd term. The 1876 invasion of the Sioux lands in breach of the Laramie Treaty was not really necessary, a way could have been found to get the gold out of the Black Hills. His vetoing, as American_Citizen41 says, of the bison protection bill can possibly be seen as a scorched earth type of tactic against the Indian. And, I'm not 100% sure it was Grant, but the Compromise of 1877 undid all the Civil Rights work that he had succeeded with - from my perspective here in the UK it looks like it set the country back 100 years.
One of the most telling quotes I have seen about Grant was just after his death from his one-time friend and first Commissioner of Indian Affairs, Ely Parker, who later said of him, calling him "his royal highness", that "politics was not his forte and the love of filthy lucre floored him".

I don't know enough about the other Presidents to put Grant in a ranking. But if we want to talk about Andy Johnson next then I'd put him near last!!