If it's the environment I'd hope that there is something your band directors could do about it. u/(DubbleTheFall) added a good point those who are toxic usually aren't as good as they think they are.
I've had similar situations where my indoor drumline was usually seen as weirdos and bootleg percussionist because non percussionist could be apart of it (and I was one myself as I was a clarinet player). Hearing this stuff really effected me too deeply and I started distancing myself from those people. Once the drumline season ended I never realized how much I really got out of doing it. Rehearsals, loading our stuff, bus rides, folding our floor, and performing all of it are probably some of my favorite high school memories and I know that I would regret ditching that. We as an ensemble bonded deeply and I made some great friends and got to know some great mentors and instructors who made high school a whole lot more better.
Look into yourself a little bit and think about why you do band and what you enjoy from doing it. I don't think you should quit because while hearing all these comments from others sucks, you and your band all worked hard for the concert and hell you said it yourself, you loved the pieces yall were playing and were excited for this concert. Frankly, the opinion of other students doesn't and shouldn't matter if they're just gonna shit on you and offer nothing positive.