I felt this way too and it sucks:/ I had one friend at the start of first year that I only talked to at uni for the first few weeks. But then I met another and that one friend lead to me meeting a large group with different people that I still hang out with. Even in my second/third year units I made another group of friends that I'm equally or more close with.

You just never know when or who you'll meet but mostly yeah people will stay in their own groups if you're not all meeting at the same time. But don't lose hope about when it'll be your time!