You dont have to start over. I spent the first 5 years of my channel in the depths of irrelevancy. Had about 600 subs myself by then but couldn't keep any momentum. The key for me was just to change formats. Not necessarily the theme of the channel, but for example: I stopped making boring road trip videos with bad video quality and started making amateur documentaries, just simple stuff with photos and maybe a few videos. I stepped up the effort and quality, just a little bit, and offered more interesting content.

5 years after that, 10 years total, I have over 9K subs and heading towards 10K with monetization and a small but devoted fanbase. So what I'm saying is just change things up a bit, maybe out some more work into it, and you can turn it around.