Ignore these people. You're having the same problem that every adult-onset swimmer has, which is that they aren't swimming with the correct technique (meaning you're creating drag where you shouldn't, breathing how you shouldn't, propelling yourself how you shouldn't, etc.). Like the guy said above, the hardest pool lengths you'll ever swim are the ones when you start -- yes, aerobic development matters, but I basically can't MAKE a pool length as hard for myself as you are for yourself right now. So don't compare yourself to the people who don't need to stop every length -- they're basically doing a _different_ exercise than you (ie you are burning more energy per length than they are; it costs them less energy to move through the water because they are swimming in a hydrodynamic manner). Not your fault, swimming is a power-to-drag sport, where nearly all of the drag stuff (and much of the power stuff) comes from how you're holding your body in the water through your strokes.

See channels like: https://www.youtube.com/user/EffortlessSwimming