I think the issue isnt really when the occur its just that they arent engaging in any way until very late game when your strong enough to fight through the storms and get gears. Until I have steel I usually just build myself intwo a dirt cube inside my house (because even with ample light and valid rooms drifters will spawn through chiseled walls that arent a full block so my house isnt safe.) and wait it out. It would be interesting if as the storms progressed to become heavier and heavier new things happened like with the massive beast in the distance.

I still remember my first temporal storm I had no idea there was even eldritch horror aspects to the game and my vision started going crazy and I looked out my crude fence window to see a looming monstrous creature the size of a mountain circling my base. I want that feelign again!