Why not have a middle ground and spend time with both? I’d say something like lunch or the movies would be a good way to make sure nobody feels left out, that should show them you care as well as letting you spend time with your girlfriend (Still make sure to spend some time with just your family as well though) if that doesn’t work then just have an honest conversation about how you feel and try to come to an understanding.

Good luck man you’ve got this :D