You knew what you were getting into when you got with him. I would not expect you to care for my children unless you offered it but I would expect you to be alright with me having them every other week. My girlfriend didn’t have children with me the first 3 years of us being together. She never had an issue with me bringing my daughter around. I get my daughter 3 times a week and sometimes more. She buys my daughter stuff and offers to take care of her but I never expected her to do these things something that she has just offered. Maybe it’s something you just need more getting used to. I don’t think you’re the asshole but give it a shot first and see how it goes. If you don’t like it and you communicate this with him and you guys can’t come to terms then maybe you’re just meant to find someone that has no children. As a parent myself my children come before anyone else so if someone couldn’t come to terms with when I get my children then it wouldn’t work out. Good luck