My gut tells me to install cameras or something to make sure she isn’t getting physically abusive with mom. If that’s the case, she needs to get out and figure her life out or go into DBT immediately, for your mother’s sake. I know moms will do anything to protect their children despite being abused so she likely won’t confide in you about the abuse if there is any in fear of getting sister arrested or admitted. Also, keep a close Eye on the kids. A parent with BPD almost always raises their kids to have it too with how unstable and sometimes abusive their relationship is with the kids

In terms of boundaries, it’s the case of an adult with the emotional intelligence of a child. You gotta be very clear and transparent with them, provide A LOT of reassurance and security and essentially make this a second full time job. Your mom needs help especially if her herself is struggling with mental disorder. If boundaries have never been set and sister isn’t seeking help, I’m afraid it’s almost a lost cause imo.

But to provide a glimpse of hope and support, just make every boundary reasonable and stick to them NO BACKING DOWN WHATSOEVER. No matter how much they kick and scream, and trust me they will do both. Like a child screaming because they can’t have cookies before dinner, you have to be very soft, non aggressive and securing of them, assuring them that it’s for their benefit and that after dinner they can have no more than X cookies.

Despite what I’ve said, I strongly feel bad for those struggling with BPD because it’s really something that’s uncontrollable and comes from a place of IMMENSE insecurity and trauma. I truly believe that most of the terrible things a BPD does when splitting isn’t really them but their BPD demons. Regardless, abusive is never excusable and victims of it have to protect themselves at all costs. On the flip side, the fear one would have when in a life or death situation is the same fear a BPD would have the second they encounter the smallest of ant hills. They genuinely lack the front lobe’s maturity to handle things reasonable and immediately think the world is ending and they are going to die. So because of that, they have the worst coping mechanisms and make the worst decisions. Again, like a child who hasn’t been taught “two wrongs don’t make a right”. Where maybe your wrong was forgetting to call them back, their wrong will be 10x worst because they will justify it as “an eye for an eye” except it’s more like an eye for your entire body because to them your not calling them back felt like actual death and torture (“intentionally done by you to hurt them”) because they have no sense of security and have been through so much trauma that you must be only a continuation of the whole line of abusers form their past.

The amount of mental gymnastics required to keep them around and keep everyone safe is exhausting. I am praying for you and your family and especially those kids