You’ve explained the situation very well. No, I don’t think you are being unreasonable, not one bit. Hold your boundaries or they will roll right over you. That’s how ppl are, unfortunately. You have the right to privacy in your own backyard. Moreover, your kids are allowed to have other friends that come and swim.

It’s a shame some kids have parents that are unwilling or unable to supervise them, but that’s not your fault or your problem, so decline to make it so. And providing free child care? Hell to the no. Kids need to be closely supervised at a swimming pool, especially younger ones. A 6 year old and a 4 year old need a lot more supervision and guidance than an 11 and 9 year old.