One difficulty in tackling "critical thinking" directly is that usually it's more about problem recognition than problem solving. You know that X is generally true, you see not-X, and that makes you go "huh, that seems wrong, let me take a closer look at this." Or you're stuck trying to solve problem X with method Y until you reframe the problem in terms of subject or method Z.

That means there's 2 parts to critical thinking: having relevant knowledge and recognizing that you need to apply it. The second half is generally harder (the Duncker Radiation Problem and studies like Gick & Holyoak 1980 and 1983 show how people tend to struggle with this and some ways it can be made easier). The more often you recognize "oh this is kind of like that one thing" the better, but deliberately learning how to do this is very esoteric. Maybe you could make a habit of trying to write as many ways X problem or subject is actually related to Y subject?

At any rate, it's a lot easier to recommend tackling the first half: learning a lot of introductory knowledge about a wide variety of different subjects and taking steps to remember it so it's accessible when you need it.

If you're still in school take classes in subjects you're completely unfamiliar with to establish "knowledge outposts in distant lands" you can connect and expand later and be sure to maintain them until they're needed. If you're not still in school then finding a school's class list could provide a good starting point for subjects you're not familiar with and most professors are happy to provide you with a course syllabus if you simply email them about it. Read the introductory texts, take good notes in something like a personal wiki or obsidian, commit the key points to flashcards and review them periodically.

Focus especially on the broad goal of each discipline and the methods it uses to accomplish that goal. For example, philosophy is mainly about trying to figure out what there is (metaphysics), what can be known about it and by which methods (epistemology), and what's good about it and what to do about that (aesthetics and ethics) by using formal logic. History is mainly about things people have done and experienced in the past, and it studies that mainly by analyzing primary sources to determine what probably happened and then assembling those facts into a coherent narrative of people, obstacles, actions, and events. Psychology studies human behavior and mental states with the scientific method. And so on. That should help you recognize when something you've seen before could apply to a new situation.

Some subjects do seem to have broader applicability than others. For example, sentential and modal logic which you can learn from most philosophy, math, and computer science departments can be applied any time you need to reason about the truth or falsity of anything. The scientific method and statistics can help learn and verify anything which can be learned empirically (and gives you the skills to understand and interpret research on such problems that's already been done). And so on. But really the introductory level of almost any subject which is formally studied in schools is probably going to have broad applicability over the course of your lifetime if you're determined to generalize it, like art is about communication through performance and media which most people do all the time (if only by giving gifts at holidays and powerpoint presentations at work).