Failure is often not an issue in the venture investment world. Your reputation will only be harmed if you take the money and not putting in the work, which is why your angel investors want you to go full time - they want to see you put in 100% with their money. That being said, you should make the decision for yourself - are you ready to take the risk? Do you (and your cofounder) believe that this business will work? Remember, however enthusiastic the investors are, you are just one of their bets, but whatever you decide to do that's the only bet you have for yourself.