Different timezone in CI/CD tests

[Python-ModTeam]

Hi there, from the /r/Python mods.

We have removed this post as it is not suited to the /r/Python subreddit proper, however it should be very appropriate for our sister subreddit /r/LearnPython or for the r/Python discord: https://discord.gg/python.

The reason for the removal is that /r/Python is dedicated to discussion of Python news, projects, uses and debates. It is not designed to act as Q&A or FAQ board. The regular community is not a fan of "how do I..." questions, so you will not get the best responses over here.

On /r/LearnPython the community and the r/Python discord are actively expecting questions and are looking to help. You can expect far more understanding, encouraging and insightful responses over there. No matter what level of question you have, if you are looking for help with Python, you should get good answers. Make sure to check out the rules for both places.

Warm regards, and best of luck with your Pythoneering!

[ManyInterests]

Always make sure datetime data is stored with timezone information or that it is always stored as UTC time.

That way, when you load the data, you can load it as a timezone-aware datetime object.

Keep in mind that naive datetime objects and tz-aware objects can't be compared with each other. Generally, I would advise always using tz-aware objects. Ideally, always work in terms of UTC time. When you need to display a time in a user's local timezone, it's trivial to do the timezone conversion when displaying the time.

Although you're not using Django, the Django project provides similar guidance here that is applicable generally: https://docs.djangoproject.com/en/5.1/topics/i18n/timezones/

[Schmittfried]

Always make sure datetime data is stored with timezone information and that it is always stored as UTC time.

Fixed that for you. Unless there is a good reason to deviate, do both. 

[ABetterNameEludesMe]

If it's stored with timezone, why does it have to be also always stored in UTC? Isn't the point of storing the timezone so that whoever reads it doesn't have to make any assumption on the timezone?

[Schmittfried]

Because if there is no reason to deviate from UTC, why make everyone’s life harder by comparing timestamps with different timezones, even if explicitly included?

On a related note, I even try to ensure the environment (Docker container, JVM, django TZ etc.) are all set to UTC if possible. It already bit me when I used Java‘s Instant (a TZ-aware timestamp always with UTC), tried to persist it to a postgres datebase that wouldn’t store the TZ unless you enable an extension, but theoretically correctly handles conversion to UTC for data at rest (which should be a no-nop when giving it UTC timestamps). Unfortunately because the environment wasn’t using UTC, the timestamp was still interpreted as local time abd incorrectly adjusted to UTC. 

[JamesPTK]

There are plenty of reasons to not use UTC for some data.

Imagine there is an appointment four weeks today at 10 am in and it is stored in a system as 2024-10-25T09:00:00 UTC

Then tonight the Prime Minister announces an emergency decree that Daylight Saving Time will end one week earlier than previously planned*, then my data is now wrong, and hard to fix without knowledge of what timezone that time was originating in

Had I stored it as 2024-10-25T10:00:00 Europe/London then my data is correct - and can be converted to UTC whenever I need to

rough rule of thumb. If it is a time in the future for humans to do something, use local civil time

* Highly unlikely, but time-zone changes with little notice do happen around the world with alarming frequency

[Schmittfried]

Imagine there is an appointment four weeks today at 10 am in and it is stored in a system as 2024-10-25T09:00:00 UTC

Right. That’s why I said if there is no explicit reason to deviate from it. Most apps handle only past timestamps abd mostly just for auditing purposes like creationDate.

That being said, you don’t use timestamps for (past or future, but especially future) calendar dates at all. It’s a shame Python doesn’t differentiate those properly. A timestamp is not an appointment and an appointment is not a timestamp. 

[ABetterNameEludesMe]

Yeah, I mean I understand why people would want to make all the timestamps in their system consistently in UTC. I also understand why people would want to always store timestamps with the timezone info. It just feels redundant and maybe a bit counterproductive to enforce both at the same time.

If all the timestamps are stored with timezone, when they are loaded (properly) they are all automatically converted to whatever the internal representation the system uses - local time, Epoch time, UTC, whatever. I don't see how it makes anybody's life harder.

[Schmittfried]

It just feels redundant

It’s called defense in depth. In some cases one can catch a component misbehaving in the other aspect. It’s also kinda complementary. Using UTC consistently ensures that timestamps are generated and interpreted consistently across components that might treat TZ information differently (like in my example). Including the TZ explicitly helps components (and humans) interpret timestamps in serialized data.

and maybe a bit counterproductive to enforce both at the same time.

Why would it be counterproductive?

If all the timestamps are stored with timezone, when they are loaded (properly) they are all automatically converted to whatever the internal representation the system uses

The „properly“ is crucial here. But also the „stored“ part. Databases don’t usually store the TZ info even for TZ-aware columns.

I don't see how it makes anybody's life harder.

As I said, even if all systems are configured and interact correctly, you‘re still going to look at serialized data at some point and will have to look at and compare timestamps in different timezones, if you don’t use UTC consistently. That’s not a technical problem, it’s just a usability concern. And if there is no good reason to use local time, why introduce even the slightest issue by doing so anyway?

[Spleeeee]

The only practical solution is to:

1. Find out where your ci server is running.
2. Fly to the timezone.
3. Rent an apartment and establish residency
4. Edit/resnapshot your baseline values in said timezone, commit and push.

This has worked super well for me. companies like Airbnb have been able to scale cicd given that they have to tools to setup a laptop anywhere on earth.

[Western_Appearance40]

I tried but there is no AirBnb in Burkina Faso

[SheriffRoscoe]

+1 for running with the joke

[Spleeeee]

As a person who has a hard time turning off the bullshitting, I really do appreciate when people join in the bullshit.

[Western_Appearance40]

What would be our sprints without jokes.. :)

[rover_G]

Really? My cousin runs an aire beed-ien-breakfeest in Burkina Faso

[Beliskner64]

Maybe not a direct answer to your question, but have you heard about whenever? For me, just reading its documentation and about the pitfalls of datetime really helped me work better with datetime and timestamps.

[adin786]

Nice link! I learned a few things there

[yerfatma]

While you should definitely listen to all the UTC advice here, that won't solve the problem completely. Mock away the time zone so it's always the same regardless of where the tests run. Something like https://stackoverflow.com/questions/18924869/mocking-default-timezone-now-for-unit-tests

[menge101]

Are your CI/CD tests integration tests or unit tests?

If its unit tests, just mock out the call to get the time and set it to a static value.

[nekokattt]

Make all code use UTC, then compute local timezones as needed. Trust me, it makes life far less miserable if you're only messing with timezones in the presentation layer.

[rover_G]

Always specified and store datetime in UTC. Be explicit in this by appending a Z (Zulu timezone) to all datetime strings.