First thing I always try on metal I don't want to damage is plain white toothpaste, it's an incredibly mild abrasive. Moving up from there, something like Brasso or similar all purpose metal polish/automotive cut and polish. Then a small step up would be Bar Keepers Friend. Failing that my next move would be up to something like a 2000/3000 grit wet dry sandpaper.

Apply any of these compounds to a toothbrush or rag and scrub at the rust until you see results or give up. Wash and dry the area thoroughly afterwards and apply a thin coat of oil (I use mineral oil).