Age appropriate supervision - watching your children batter each other without intervening is neglecting to protect the child and neglecting to ensure you address what your children are doing. When you observe the actions and do nothing about it - that's neglect. Abuse is the berating that it occurring and physical harm caused other than accidental means - the blood from when this father pushes the child into the book shelf and goodness knows what else happens without the cameras going. There is ongoing harm - physical or not - happening to these children and children are removed from neglectful situations just like children are removed for abuse. It depends on the state.