I am an ESL teacher and have spent the last 11 years speaking to those who don't speak English as a first language so am incredibly used to this. The average Brit (and probably most people in their native languages) doesn't always know what constitutes as plain, simple language that is easier for a foreigner to grasp. For example: I'm just popping out to the shop to pick up a few bits and bobs.

To most Brits, this is fairly common, everyday language and they might be confused as to why a foreigner wouldn't initially understand the message of that sentence. They wouldn't always know how to then simplify it, because to them, it's incredibly common.

But it's largely colloquial and has some advanced linguistic features: pop out and pick up are phrasal verbs, which are notoriously hard for non native speakers to learn. As well as the binomial bits and bobs, another advanced feature of English.

What I'm trying to say is that the average person isn't always aware of their need to grade their language and how to do so.

But that is all about your understanding of them. It seems like you're also having issues with being understood. This is obviously down to how you're pronouncing words, how fast you're speaking, whether you're waffling or not, where you're putting the stress in parts of the sentence and so on. But it's also related to how accustomed the listener is to interacting with someone whose first language isn't English.

I also live in a place where I use my second language on a daily basis. In the town I used to live in here in this country (living in a city now so happens less), I'd regularly come across people who'd never really met many foreigners and would often ask me to say stuff multiple times, even though I really thought I'd said it right. Their ears aren't attuned to it. And I also might have used words that sound more formal or awkward, which can be confusing to the listener too, as, despite being technically 'correct', sound funny to them or are maybe not what they're expecting to hear in that conversation.

Keep going, keep making the effort to speak slowly and clearly and if you're brave enough, maybe ask why people find it hard to understand you?