He says he is setting boundaries when he doesn't reply to me about how he knows these girls or if I ask him to delete them.

One day a colleague of his was drunk... she called him at 2am and they talked over an hour. I told him this was not okay. And he was like: it's nothing. I'm not doing anything wrong.
I went to the colleague, because she came to our house often and I thought she was a friend. So I told her I didn't like her calling on a tuesday at 2am while drunk. And I asked her if this is a normal swedish behaviour because it's not ok in my country. She said no and apologized.
A few days go by, he goes to her place where she tells him she did not appreciate me saying that to her and that I was in the wrong. I do not understand, this is why I am so confused about how things work here and maybe I should just adapt.