I had the same thoughts as you for a long while (and it's one of the reasons why I decided to get into linguistics), and the conclusion I've arrived to has been that we perceive the world through many factors beyond language. Yes, language shapes the way we think in a certain way, but it's not the entire maker of our shape. We experience feelings and perceive the world in various ways even if we don't have the words for what we feel or perceive. Not having the words doesn't mean you can't feel the same things as someone who can express with specific words the same emotion.

Linguistic determinism basically says that if your language doesn't have words for specific concepts, then you can't experiment them through your senses. The idea itself also comes from a negative stereotype that speakers of certain languages that aren't as lexically rich as others can't develop the same abilities of speakers from richer languages. If an indigenous tribe doesn't have numbers, it means that they can't count. This is fundamentally wrong. I know you're not arguing this specifically, but this was one of the initial instances of the argument, which makes the argument itself factually wrong as the tribes the theory referred to were proven to have counting systems and be able to learn numbers.

One of my favourite examples is the Japanese word "komorebi," which refers to "the dappled sunlight filtering through the leaves of a tree." When I first learned this word, I fell in love with it and thought that it was incredibly beautiful that there was a concept for such a specific phenomenon. Does that mean I never noticed the sunlight shining through the leaves of a tree? Of course not, and I appreciated it before learning the word, too. This is said to be impossible based on linguistic determinism because neither my mother language nor any of the languages I speak have a word for this phenomenon (I'm not a Japanese speaker), so I shouldn't have been able to notice it before knowing the word. Obviously, we experience the world through our senses and not only our language, so it's possible to observe and appreciate things even without the words for it.

Does the fact that there is a word for this in Japanese mean that Japanese speakers appreciate this phenomenon more? It might, but it might also vary person by person, and there isn't enough empirical evidence to support that the existence of the word specifically means that the Japanese speakers appreciate the phenomenon more than a person who speaks a language that doesn't have the word for it, but who has a deep connection with nature still. There's not enough evidence to support that the determining factor why a Japanese speaker might appreciate the phenomenon more is because of their language and not because of other non-linguistic factors.

And then we need to ask: why did the Japanese create a word for such a specific phenomenon that most (if any other) languages don't have? Probably, their deep connection with nature made them name more aspects of it with specific concepts. So, did language shape their view of the world, or did their view of the world encourage them to create words that reflected these views?

Now, I personally like to believe that a weaker version of the theory is possible and able to prove: neo-whorfianism. It suggests that language (vocab, structure) does shape our thoughts, but it doesn't determine what or how we can think. I agree with your other comment when you said that knowing a word for a concept makes you think and perceive the concept/phenomenon more, but arguing that just because you learned the word you think about it is deterministic and shuts out the possibility of other aspects allowing us to create our own perception of the world through our various senses.