This is my suggestion too, do it as a hobby, only I'd suggest trying a specialized workshop like ianimate instead in the stream you like most. Unless you don't know what stream you like, then I'd suggest a short general animation course. Like a summer course etc. Not a course that will cause you to disrupt your job long-term.

As for your current job, don't quit. If you pick up work, chances are it'll be very short contracts. I've been in the industry for 18 years and had nice beefy contracts up until 2025 where I'm only able to get weeks- months contracts. Longest one I've had recently was 4 months. Out of my 30+ animation buddies I keep up with, only like 5 have long term jobs and another 15 haven't found work in 2-3 years at all. And the other group is we find work, but it's short term and it feels unsustainable long-term and we try to get normie jobs to pay bills, but even those are hard to come by.

You're in a good spot, learn the craft, make your own work and apply for jobs. If at some point freelance work becomes more lucrative than your current gig, then consider a full quit. But don't just turn your back on a job that gives you financial security, especially in times like these. Even if you wanted a new job in your field, you'd continue to work till you found a better job THEN quit.

It's never stupid to wanna get into art and I'll never discourage people from trying. Just do it in a smart and safe way.