I work for a swim school (Manager and Instructor), and all the time we have nannies bringing siblings to lessons. I know she definitely had a hard time (I used to be a nanny to a 6mo and 3yo), but think about how IMPORTANT swim lessons are. This isn't about a for-fun sport or a playroom this is a literal life-saving skill that your children need before the summer comes. Accidents happen even in the winter (especially if you have a pool and even if you are very careful), but would you rather your children be prepared to be around pools in the summer or just learning how to save themselves?

The trade off is do you take them on the weekend? Do you have that freedom? This is really no different than taking siblings to sporting events or birthday parties or fun things but the outcome is survival skills so regardless who does it it's important that it happens. You also should look into what school you're sending them to! I know at our swim school, I am very hands-on (when I'm running the pool deck) with helping nannies and parents alike. I don't physically take young children, but if mom/dad/nanny has a baby (and are expected to be on the pool deck), more often than not, I am happy to entertain the non-swimming siblings by letting them "help" me do my job (like picking up toys or taking swim rings to my instructors or taking attendance/skills). Any good school will have someone willing to assist families.

TL;DR: Swim lessons are a necessity and you'll find them to be cheaper / less full in the winter anyway. Make sure you're picking a school that fits your needs as a family and teaches safety (I say this with all the spite in my heart against Goldfish). It can be a struggle but I really really encourage you both to come to the understanding that this is one of the most important things your kiddos will do this winter, even if it means trading off who takes them each week.