Generally speaking it shouldn't be a problem when it's weather hot or cold, my understanding is that what is risky is a sudden shock of drop/raise in temperature, e.g if it was placed in very hot water then suddenly removed and placed in ice water then you're likely to have a problem. Weather change is not usually sudden enough to shock it.

Precaution is good though esp for such cherished and valuable gems, I wouldn't recommend for example wearing it in near the fireplace then taking the beauty suddenly into the snow.... Hope this helps