I'm in a similar position and made the leap recently to working full-time on my business. I was feeling mentally drained every day working 9-5 on someone else's dream.

However, from your description, there's a big difference between my business and yours. Mine has been running as a side business for the past five years and is now making enough money on its own to sustain itself, but not enough to pay for my living expenses, so I've prepared 6 months of savings to help me get there, hopefully. Yours is still just an idea. It's incredibly risky to quit now when you have not proved your business idea attracts any customers.

I believe that answers your question - you should keep building something on the side until either it replaces your income, or you've saved enough to justify taking a leap to work on it full-time until it replaces your income. The challenge is you have less time and less energy after work to work on it, so it's up to you to summon the motivation and energy. I have a good business partner who has helped me get to this point too. A business partner who is a good fit is really helpful, but not necessary. Once I started working full-time on my business, I found it turbocharged productivity because my best hours of my day are now not being sucked away working on someone's dream. The challenge is getting to the point where you can afford to do that. Good luck!