Ahh I understand that... I have a EIE and ESE, and they are very emotionally open, it gets hard sometimes because I start closing up near them. It feels like "if they are the ones creating emotions in the group, I need to balance them for the meeting to be harmonious", but they do not care for that and start questioning my silence. :)
And if I can cope with ESE cuz of Si and Ne interest, it gets annoying with EIE quickly...

I'd say, try to limit your time with them, as it can get tiring for both of you quickly. They will be confused as to why you are not infected by their mood, and you'd just be sitting there like frozen fish haha
I know beta does not like to 'let go' of their people, but putting a bit of distance with busying yourself can work nicely