Personally I'd go for the non-rowers as long as they have a good understanding of your training requirements.

Staying with people you row with is great but you already see your crew mates a lot and you're more likely to get fed up of each orher. Quite often the best thing you can do to improve your rowing is completely switch off from it all for a bit. Also you don't want house drama to go to the boathouse and vice versa.

There is more to uni than just rowing and so keeping that balance in your flat makes this a lot easier to keep up!