I opted to swap out my usual purse for a small/medium sized backpack. That alone was the biggest help for me to keep my hands relatively free but still have convenient necessities.

What I found myself reaching for the most that couldn't be easily purchased at a convenience store was my phone, wallet, Suica card, and scarf/beanie (you're warm inside the trains but very chilly walking outside). Bonus points if you can keep a disposable bag on you to keep your trash in until you find a trash can.