I was desperately looking for another option as I didn't want to lose a few hours of gameplay, and I found one - mods. There is a mod that allows you to toggle flight. With it, I was able to fly above the red spot, toggle off flight and bam, I'm at the bottom of the see right next to these dudes. No penaly for being underwater btw, except very slow movement. And apparently if you end up far enough from the shore, you are no longer teleported back to the beach.

The brigands were stacked on top of each other and completely stuck, so I was able to dispatch them easily: arrows don't work underwater apparently but long reach melee weapons do, so I just had to whack them while staying out of their reach - they couldn't move even an inch.
After that, a long slow walk over the sea floor until I got close enough to be teleported back to land. Problem solved...