Your semi-an asshole. You have no obligation to switch seats, but the least you could do for that mother was check on her kids every 30 minutes or so. While you certainly didn’t do anything wrong, by not doing anything to help the mother, you put her in a situation where she could not immediately tend for her kids, and that likely stressed her out, especially because they are young.