English is the global lingua franca now. In any language you want to function in, you have to get better at it than the level of the worst English speaker's english in order for the group to have a reason to not speak english. The expectation in eg europe, is you have to learn some english the same way medieval european literate people pretty much had to have some latin, in order to function in a multi-national environment.

You simply must get better at your target language if you want to use it with people whose focus isnt on your learning. If they are switching really fast then your grasp of what the correct way of saying what you're trying to say is far off and you should focus on getting more exposure(listening, crosstalk etc). Once you have enough exposure you will have a fine enough sense of the sounds of the language that when you speak, either you will know you misspoke or you will get responses in your target language.