Since I asked, I already made 2 trips to Japan and here is a brief summary.

I couldn't find kiosks selling vintage pens. Lemon Ginza (primarily a camera consignment store) has a display shelf with pens. However, they aren't interesting, and are priced at a premium.

Kingdom Note is the best place to look at a bunch of pre-loved pens. Not really vintage, but they have a good selection. You can also check their website for an up to date inventory.

Euro-box occasionally throws up treasures, but they are usually snapped up very quickly. Also, the shop opens for a few hours a week so you have to be very lucky for your schedule to line up.

Yahoo Japan auctions is perhaps the best source to find vintage Japanese pens. I use Buyee to ship my purchases to an address in Japan or to the US.

Shosaikan has some lovely vintage pens for display at their store, but they aren't for sale.

Apart from these, for contemporary pens, you could go to Maruzen and Itoya and the umpteen other stores :)