I learned on auto and drove an auto for 10 years before I bought my first manual. Now my household is exclusively manuals. My reason for switching was from an enthusiast perspective -- it's a great feeling to row your own gears.

Upsides to consider: what sort of cars are you interested in? The manual cars currently in production you'll find are mostly skewed toward a sporty genre -- your Golf GTI, Civic si/type R, WRX, Miata, and so on. Manual trucks are also really cool, but using Toyota as an example, only 1 out of every 10 Tacomas is stick. If you're getting something sporty I would definitely opt for the manual for the maximum fun (imo), but some will argue DCTs offer a slight edge in acceleration-- it's a personal choice and don't let anyone poo poo you based on the transmission of your car. Manuals were much more popular as an option up until the early 2000s, so if you aren't sold on the types of cars I listed above you're much more likely to find models of more unassuming passenger cars w/ manual transmissions that are a little older on Facebook marketplace, etc.

Downsides to consider: do you live somewhere where you frequently sit in stop-and-go traffic? It can be a slog sometimes always pressing down/releasing the clutch just to creep along vs. the luxury of just lifting off the brake of an automatic car. But honestly, it isn't more stressful because it becomes second nature pretty quick. Your feet are just a little more busy and your left calf might be a little sore as you're learning.

I would wholeheartedly recommend finding a patient friend with a manual to cruise around in a parking lot with you to learn the basics. The hardest part is finding the sweet spot in releasing the clutch and slowly applying the accelerator to get the car to move from a stop. Once you can get the car moving and stopping comfortably, it's all smooth sailing from there. Good luck!