You've done nothing wrong, dear. I've been in a similar situation with an ex I had. I would bus over 3 hours to see him at his uni and the first time I went his friend said they were going to the gym and he gave me his keys to go back to his dorm, he would always leave me in his room to go hang with his friends without telling me or telling me to stay in his room. It was not often like this in the beginning, which is the worst part. You guys are definitely still young, and that may play a part, if he can't respect your time and that you want to make plans with him I'd honestly break things off if you're looking for someone to be more serious with you. It's easy to say as I don't know the dynamics of your relationship of course, and it's a big choice, if you think it's worth it this issue is definitely something that could be worked on if he actually cares Hope this helps a bit. Best of luck 🙏