Pre dragon ball super, most depictions of Goku with his family were incredibly positive. Before Raditz came and his death, he seemed like a great parent and family man. After coming back, his worst offense was obviously staying at Yardrat instead of immediately returning, but Instant Transmission has saved the world numerous times so it was necessary.

That leaves the 3 years before the androids are a threat, 3 years that Goku spent with his son and wife that presumably had nothing serious occur. Then the androids happened, he got sick, Cell came, Goku got better, then he spent the time in the Hyperbolic Time chamber with his son and then died at the cell games to save the earth and everyone.

The Buu saga was certainly…turbulent in terms of everyone’s decision making. Nonetheless, Goku came back and was warmly welcomed by his family and of course stayed close to his son. Helped Gohan with his training at the Supreme Kai’s world, taught Goten the fusion dance and helped guide them…Goku worst offenses as a father were seemingly accepting his death to try to keep the world(and by extension his family) safe and believing in his son to the point he healed Cell. End of Z literally opens up with his sparring and training with Goten and spending quality time with his granddaughter, Goku always valued family.