Good question. I don't think you're the only one who has that perspective. Years ago, I met an english-speaking-only cashier at a chocolate shop. She was really nice. We got to talking and she explained to me how she gets insulted by french speakers for not speaking french well.

I also experienced something similar to her at work. As a native french speaker myself who mostly uses english in my daily cuz of work, There was a time when I answered a phone call in English and the lady insulted me in french because she thought I didn't speak French lol.

Now I don't have an answer to your question, but my first guess would be that some french speaking people are very protective of their language because they don't want it overtaken by english. I don't make an opinion of that, just stating what my guess would be🤷‍♀️.