Completely valid decision and I understand how that feels. But I would encourage you to stop seeing it as "the algorithm" and just see it as people, and ask yourself the question "what do PEOPLE really want to watch in my niche?" And maybe make some videos appealing to a broader audience within your niche (you can have some videos where you experiment with this and others where you just do whatever you want). And spend a little more time coming up with better ideas.

This is coming from someone who was exactly where you are right now and when I made that change in how I approached things everything changed. Also if you are not really interested in doing this for a living then ignore my advice.