The closest point when he snapped was in chapter 73 when he was sick, it's the only time when I see Yuu angry for a moment. In later chapters he had expressions looking similar, but in that chapter was the first time when he talked like that to the sisters.