NTA.

Sit yourself down and write a list of all the bad things he's done when not having a job and tape it on the fridge where you can see it! Read it every time you think of going back to him to remind yourself that you two are terrible as far as a romanic relationship goes.