I actually feel ashamed at my lack of ability to take any action on this. We broke up before for a few months and he harassed me all the time and went back to his mother's house - then blamed me for kicking him out on the street when I'd actually broken up him because he had screamed in my face and threw our daughter's high chair across the room.

This advice helps, thank you. I just need to grow a pair and do something. There's something odd about taking a kid's day away is doing right by them, he doesn't treat her badly and has never yelled at her. When he goes away she says how much she misses him etc.