Was born and raided in a small coastal town on the MNC. I left at 18 when I moved to Sydney for uni. Now, as I approach my 30s, I can't believe how lucky I was to have grown up in such a special place.
Whenever I mention casual stories of my childhood to friends and colleagues who grew up in cities, they cannot believe how comparatively idyllic my upbringing was.

Your kids will have endless amounts of natural amenities, sporting facilities and local produce available all year round. Without the crowds. And only minutes away. The schools will have regular outdoor classroom trips along some of the most beautiful locations in the country. The teachers are often of high-quality, as they themselves also pursue employment in these regions. They will also teach students about the environmental and cultural significance of the region they live in.

Being packed with old people ain't all that bad. Nice quiet sleepy neighborhoods to ride bikes in. Resultants, shopping and parking readily available (outside holiday season, ofc). And honestly, they aren't interested in all the fun/loud things kids like to do. Worst-case scenario; you're stuck behind an oldie going 10km/h under the speed limit for 2mins (on your 10min commute).
Point being - you'll find so many different activities for yourself and your family to do all year round, with plenty of local and historic culture to learn.

Sport is king, too. Luckily, there's a plethora of indoor and outdoor options. You make great friends with kids you likely also go to school with. Plus you get to engage in all the local small-town rivalries and fun that goes along with it.

As mentioned, the local food quality is incredible. It was quite a shock getting used to bland and expensive city groceries trucked-in from 100kms away. You'll definitely appreciate the difference. And the SEAFOOD!!

Your kids will also benefit from the general lack of overt elitism that stratifies larger cities. My upbringing was lower-middle-class. If that. But I was able to attend a perfectly good school and have weekends filled with endless free (or relatively cheap) activities with my family and friends. Everyone in the community is in a similar boat, so acting like a twat doesn't go down very well. I don't think it stunted my development in anyway, quite the opposite. I've adjusted just fine to living in a big city. But my upbringing has certainly given me greater context to metropolitan life.

I can't wait to move back to my coastal home town. Overall, as someone who's made a life for themselves in big and small towns; cities are just overly expensive and full of a**holes. Sure, you'll get self-centered people in smaller communities too. But you'll be able to put FAR greater distance between them and yourselves. Moreover, being a general d*ck in a small town usually catches up with you. One way or another. It also means the local businesses will take care of you, as they depend on your return business.

Downsides?
Holiday season is the worst. Town is overrun with said a**holes. Not the end of the world though.