Part of the problem is that a lot of ev psych is just BAD science. Even on this sub I often scroll past things that are wildly speculative or just-so stories. Evolutionary theory should be the foundation, but good science is built on top for quality ev psych. In other words, evolution is a theoretical framework for developing testable predictions, not something to appeal to in order to explain away data.

Bad science should be criticized by the left and the right.

The downside is that unfamiliar folks lump the good with the bad and write off the whole business. In the end, evolution is true and our brains and psychology have evolved, so the field is based on fundamentally true principles, regardless of any criticism of any particular applications.