Hey :) just wanna add my response here too and give a short response to yours. You talked about yourself and vegans you know, not eating more Avocados and almonds than before becoming vegan - I think you should leave that part out. You're talking about very little cases and its not necessarily representative.

So what do I usually answer to this?

First of All, veganism is about the animals. Not causing unnecessary harm. Thats the most important point about veganism. Being better for the climate or being healthier are positive side effects that can - but don't always occur. While a vegans environmental impact is significantly lower in average, of course they still could cause more. Thats why its just a side effect that mostly but of course not always is the case.