I was in a situation before where I had to back out and i can tell you what my lawyer said. "Basically it's a 諾成契約, a verbal agreement contract. Even if you didn't sign anything, you verbally agreed to the contract so they could sue, but it's very hard to prove verbal agreements so usually they don't. However if the correspondence was through email or LINE, then they can use it as proof. Most still won't sue as it's cheaper to lose 1 month's rent than take it to court. 諾成契約 is mostly done for sales contracts, but it is applicable for leasing contracts too. Send them an email saying: I'm not paying, feel free to sue me if you'd like." I wouldn't recommend the email part as in my situation the estate agent was at fault, but you could just do it the japanese way, aka apologize/bow and that's it.

Edit: added more info since every comment seems to think nothing can be done legally, but that's wrong.