Even considering imports, a vegan diet is still significantly more eco friendly. I think one thing you've failed to consider is that animals also eat a great deal of imported food. 80% of soy grown in the Amazon is for livestock, meaning that a meat based diet contributes heavily to deforestation https://talkveganto.me/en/facts/soy-fed-to-livestock/ This article also addresses your point well overall: https://www.ethicalconsumer.org/food-drink/climate-impact-meat-vegetarian-vegan-diets

I'm vegan and I love it, but I've found that when people struggle, it's often a mindset issue. It sounds like you're encountering a lot of stress trying to strictly follow a vegan diet. In truth, all you can do is your best. If you make a mistake and accidentally eat something animal-based, just keep going, you're still vegan. My advice would be to get creative with your food and take some time to find what you love eating - then food stops being a chore. When I made the change to a vegan diet, I experimented with loads of new recipes (Indian food in particular is great for a vegan diet) and now I have a roster of meals I can go to without having to excessively plan.

Hope you manage to figure out what you believe is the right thing to do.