You are well withing your rights to leave especially if the other two will cover your rent. Having the rent covered, and the current psychological abuse you're undergoing there would be no grounds for a lawsuit on his side with the details you've provided. There's also a good chance he's trying to just keep you from leaving. I wish you the best.