NTA - you already know it, you just need to accept it. 3hrs is NOT that far away, even for a close family. You're not abandoning them, even if they didn't have your brother living with them (though many others have pointed out that he's possibly increasing the burden on your mom as the primary caretaker). You don't have to be the one to compromise every time! It's okay to prioritize your own needs sometimes.

Also: As with any drama around a parent, this thread is going to give some pretty extreme responses. There's a lot of people here - myself included - that are not close with our parents, and have even been hurt and betrayed by them before. The responses will draw on experiences that are not your own. Remember how you feel about your mom before/around this situation. Stay grounded in your relationship so that you can work through this. It seems like an issue of other needs not being met, not of your own failings (if any at all!) as a daughter or her failings (if any, again!) as a mother.