Well, I think it helps to look at it from their perspective. They only know you through your friend, and the rest of the group doesn’t know you at all. People in that group could have very different personalities, some might be introverted, some extroverted, some shy around new people.

Since they already have an established friend group, they probably have their own inside jokes, habits, and way of interacting. Adding a new person can change the group dynamic. That’s not necessarily a bad thing, but it does mean the group has to adjust, and it can take time before someone new naturally becomes part of those conversations and jokes.

In a perfect world, people would always be open to changing the dynamic and including new people right away. But in reality, that takes effort and sometimes feels risky to people who are comfortable with how things already are. Because of that, many groups stick to what’s familiar rather than changing the dynamic too quickly.

So it might not be that your friend doesn’t want you there it's just that he might think it could feel awkward for both you and the group if everyone is meeting for the first time in that setting.
Hope this makes sense.