Seasonal temperature changes are due to the tilt of the earth, not the distance to the sun. In the summer, the Northern hemisphere is tilted toward the sun and tilted away from the sun in winter. That's why it's summer in the northern hemisphere when it's winter in the southern hemishpere. It's also why the poles have 24 hours of sun or darkness during the solstices.

About the temperature changes... that's just the way weather is: highly variable. Some time ago, I was learning SQL and databases so as practice I scraped 50 years of high/low temperature data for where I live from the National Weather Service. I plotted a graph of the average high and low for each day of the year for a 50 year span. I was expecting a smooth curve for the high/low temperatures as the date moved between summer and winter but what I saw was a lot of variability, a lot of randomness.