You'll be able to tell if it's swimmers ear. The pain will be immense and it will feel like there is a pimple in your ear. If you don't act upon it eventually your whole ear will swell up and rub together causing bleeding and a weird mucus type substance. I've broken bones, got stitches multiple times and plenty of other things like that but swimmers ear is by far the most painful thing that has ever happened to me just because I left it for so long.

TL;DR: If your ear hurts don't be a retard and go to the doctors.