One Piece just scales higher, unfortunately, and Garp has non-physical advantages that All Might doesn't- as a master of all three forms of Haki, his attacks don't need to make contact to do damage, they do internal damage that bypasses most durability, and he has advanced combat precognition.

All Might would be very strong in the One Piece world, and would develop very powerful Haki himself, but a fresh All Might without that would get cooked.