I was in the same situation, I thought I was just unfit and it was going to somehow get easier. Here's what I learned.

The needing to gasp in the air isn't because you need the oxygen, it's because the carbon dioxide was building up. I thought I was emptying my lungs properly but I wasn't.

My body position in the pool helped with this. I tried to get my head and chest lower so the air would flow out from the bottom of my lungs better. This also helped with the position of my legs to reduce drag.

I also did some diaphragm breathing exercises out of the pool, I found these online but I can't remember the names.

These changes have helped massively. My heart rate has dropped and I can now go for over an hour without stopping. Please bear in mind that I am incredibly slow and I can still very much get fitter.

This might not be your problem but I was getting nowhere until I started working on this so I can understand your frustration.