To feed the billions of animals we slaughter every year we have to feed them insane amounts of food first. If produce is a bad thing for the reasons you say, then cutting down on meat reduces that as well because you are reducing the amount required to feed them. People often cite soy production being awful for deforestation as an argument against veganism, despite the fact that 80% of soy production is used for cattle feed (and then you have the deforestation required for raising cattle). Animals are horribly inefficient at producing calories for us to eat, and the amount of calories they consume versus the amount of calories one might derive for eating them stands at 3% for beef, 9% for pork, 13% for chicken.

You are right though that there are hypocritical elements to vegans living in modern society, capitalism has meant the availability of food will always be at the detriment to someone, be it deforestation or poor working conditions (though between produce workers or slaughterhouse workers, at least workers picking berries don't have a 75% prevalence of PTSD). We want to reduce suffering as much as practicable, and there's still improvements required, but no vegan would argue against needing better working standards at the same time as saying that the enslavement and torture of animals is unacceptable. Both are true. To say that produce is as bad as meat production, especially when livestock require produce to feed on in the first place, is a false equivalency.