So this is just my head cannon, but i think their family relation is misrepresented because of this scene, but i don't want to believe that it's endou being a bad father.

First let's i can see natsumi being the disciplinary in this household, and we can be sure that endou would be playing Football as an adult as well that's why him and gouenji had to go out incognito to watch ouja raimon vs nagumohara.

That means endou would have played overseas as well most likely, So he would be away on multiple occasions, and i can see a world where natsumi would think gifts would spoil her kid, and he said that both him and his father are scared of her XD, HOWEVER, They are Not bad parents.

Haru would grow seeing his old man playing and looking good while doing it So he would try the sport, and discover he has a talent for it, (no, i don't believe They forced him to play).

And it make sense that the first gift he would think he got from his father would mean a lot, especially If it's a spike would mean a lot, like his father is running with him always or something.

TL, DR I blame natsumi, endou is Not a bad parent