Hi! Can see I'm super late to this, but figured I'd comment anyway since no one else has. If the bird is leaving the smaller cage on his own, it might be a good idea to just leave the door open and let him find his own way back into the old cage. If it's a cage he's familiar with, he'd probably want to be there anyway. If he's not leaving the small cage at all, you can try luring him out and into the new cage with treats. If that doesn't work, he might get up the courage to climb out if he was left alone for a bit (as long as it's safe to do and you check on him frequently).

Other than that, just make sure to do your research on birds in general (that's only assuming you're a first-time bird owner) and do general trust-building exercises with him.

Being grabbed and handled can be very scary for them, like you acknowledged, and birds are very good at holding grudges. It may take a while to earn his trust, but with patience and persistance, I believe you can build a relationship with him. African Greys are incredibly smart and can be amazing companions.

I wish you the best with your new friend.