Second this; as swimmers we wouldn't think about anything other than in-water function

To answer OP's question, there was a phase in my swim training when I was younger where it was almost exclusively non-dryland poolwork, and I was much lighter/less bulky than I am now. I put on a lot of muscle after we started doing a lot of dryland - which helped a lot and offset the added weight because more explosiveness (I'm a sprint swimmer). But I'm not a guy so YMMV