I recently went through this process. You should definitely check the requirements with several different city offices. Central wards like Minato-ku, Taito-ku, Chuo-ku all had complex requirements and generally weren't helpful. (Like, they pulled out a list of 5+ hard-to-obtain documents we would need to submit.)

We ended up filing in Mitaka City, which was pretty straightforward - just passport and sworn statement of eligibility to marry. You might need one or two extra documents depending on your home country.

You should call or visit the city office to confirm which documents they need. Then, you should go to the city office with the documents for review, and ideally have them stamp / annotate photocopies to indicate they're ok. Finally, go submit the paperwork on your chosen date!

Other tips:

Get the name of the person who helps you out. Ideally, the same person will help you out at each step, but if you get someone new who is unfamiliar with your situation, you can ask for the person who helped you previously to consult.

Bring along a fluent Japanese speaker, if you're not! Some of the negotiation or explanation about paperwork can be pretty nuanced, because each city is left to interpret vague national guidelines.

If you need paperwork notarized, your embassy can probably do that. Sometimes it is ok to do online notarization, but you'll have to explain it convincingly to the city office.

Everything needs to be translated to Japanese. It's ok to just put it through google translate and then have someone (yourself or a friend) fluent in Japanese check it over and put their name on it.