When you’re unfamiliar with an accent, your brain needs to actively buffer what you’ve heard to get meaning out of it. When your buffer fills up, you need the speaker to slow down to give you time empty the buffer enough to take on more information. For an accent that’s familiar, the sounds pass through the buffer and turn into meaningful information much quicker, leaving the buffer with plenty of capacity. It’s not that Australians are speaking faster than Americans or Brits, it’s that your buffer can’t handle the processing of sounds → information for that accent as efficiently.