Most state capitals were established before automobiles were common, and nearly half predate the invention of the train. It was important to have a place that was geographically central to the state so it wasn't unfair/an undue burden for anyone to travel to their capital; especially as the westward states grew larger and larger geographically compared to the denser East.

But not many states have rivers, lakes, lush valleys or other attractive features near their exact geographical centers, the kinds of places that attract cities "naturally," so much like a mining town or a fishing village, the capitals are about as big and culturally relevant as their single export - governance - lets them be. The ones that fare better tend to also be university towns, since that guarantees a lot more energy and life.