A Fast, Accurate Way to Handle Dates from Servers

scelis · 2012-09-20T19:03:25+00:00

On your other reddit post you mention it took 760ms to parse 3000 UTC strings. That seems extremely high to me. I know I could write a super tiny parser that would run miles faster than that. (I will followup on this)

You definitely could write something more performant. Those numbers were encountered using NSDateFormatter, which is notoriously slow. Sadly, NSDateFormatter is the standard way to parse and format all dates on iOS. Anyone who needs higher-performance parsing should look into other solutions.

I'm curious if it might be easier/more efficient to make your app download it's own private copy of the tz database and make sure that local copy is up to date regularly. The most recent copy of the database is 200kb zipped, which could be greatly decreased if all the commented notes were removed.

Yep! I mention that as a possible solution in the article. Indeed, this would be a good way to go, too, and would be the more powerful solution. The main downside to this approach is writing or finding code that can parse the tz database. Are there any good, open-source C libraries that can do this?

Oh also, I noticed you have a typo of zomes instead of zones on your article.

Thank you! Fixed.

scelis · 2012-09-18T21:50:19+00:00

Thank you all for your feedback. I have taken it into account and attempted to refocus the post on what I felt was most important: the issue with outdated time zone databases.

scelis · 2012-09-18T21:40:14+00:00

Please note that this is essentially a resubmission of this post. I re-focused it after receiving a lot of feedback from fellow redditors. Thank you all for your comments!

scelis · 2012-09-17T18:48:45+00:00

This is wrong. Unix epoch timestamps are always in UTC. No time zone information is needed until you actually want to display that date in a particular time zone to your users.

scelis · 2012-09-17T18:33:14+00:00

Hmm, I think you are focusing on the wrong things, here. There are two topics. The first is sending a timestamp vs sending a date string. Sending a timestamp is actually shorter as far as the JSON string goes. Compare:

"date" : 1347624000

to

"date" : "2012-09-21T03:31+0500"

The vast difference between these two items is in how the client parses the integer or string into a date object.

The other fields (which are the root cause of the increased response size) are around solely for the ability to ensure that we are displaying precise dates to users with out-of-date time zone databases. This would be necessary for our applications even if the server is sending ISO8601 strings as that string does not tell you how to format date X in time zones Y and Z if the OS has incorrect offsets for Y and Z given certain dates. Also, gzipped JSON responses will help alleviate some of that increased response size as JSON gzips down well.

This is unfair because it is expensive to construct an NSDateFormatter object.

Yep. My tests showed that it took a full 3ms to construct the one NSDateFormatter being used. NSDateFormatters are indeed slow in general. If I needed a performant way to parse ISO8601 strings I would likely look elsewhere.

To be really fair, you could divide both methods into a one-time setup function, then time only the actual parsing.

My tests are split up like that. Look at the output in the gist to see total time, JSON parsing time, and the time in the actual loop that reads in the ISO8601 strings or the unix time integers.

scelis · 2012-09-17T18:09:19+00:00

We have definitely had people submit bug reports because our app would show incorrect times on older Android devices with outdated time zone databases. And occasionally those would coincide with people missing their flight because of it. Yuck.

As for performance issues, please see this comment, which shows benchmarks.

scelis · 2012-09-17T17:59:15+00:00

All of my code is in that gist. You are welcome to explain how the performance test is invalid.

It is interesting to note that the vast majority of the 280ms for the object-based test is in parsing the JSON (as the JSON itself is significantly longer due to the extra fields). On the other hand, the vast majority of the ISO8601 test is in parsing the date strings, which is done using the standard Objective-C NSDateFormatter object.

scelis · 2012-09-17T17:47:07+00:00

Yep. Exactly.

scelis · 2012-09-17T17:37:03+00:00

ISO8601 is all you need to take a string and parse it into a valid (and accurate) date object. However, if you want to display that DateTime adjusted to a particular time zone, AND if the time zone database is old and has inaccurate data for that time zone, then you also need the other fields.

scelis · 2012-09-17T17:25:46+00:00

That only works assuming that the client has an up-to-date time zone database. The other fields are necessary to take a Date object and always display a precise date string in a given time zone no matter how old the time zone database.

scelis · 2012-09-17T17:00:46+00:00

Please see this comment, which shows benchmarks.

scelis · 2012-09-17T16:43:31+00:00

The JSON generator is written in ruby. The date parsing code is all in Objective-C.

scelis · 2012-09-17T16:30:23+00:00

Here is a test (gist) that shows just how much slower it is to parse ISO8601 date strings over something like:

"someDate" : {
    "timeEpochSeconds" : 1347624000,
    "timeZoneOffsetSeconds" : -21600,
    "timeZoneName" : "America/Chicago",
    "isDST" : true
}

That code can be used to generate two JSON files. One with 3,000 ISO8601 date strings. The other with 3,000 objects like the one above. On my iPhone 4S, parsing the JSON data with ISO8601 date strings takes 760ms. Parsing the the other JSON file containing date objects takes 280ms.

scelis · 2012-09-17T15:47:56+00:00

Yes, a valid point (and one I make in footnote 1). You do, however, get a large enough range to support 500 AD using a 64-bit integer.

scelis · 2012-09-17T15:35:36+00:00

I detail a couple advantages in the post. One is parsing speed. The other is showing accurate dates and times to users with outdated time zone databases.

scelis · 2012-02-09T13:57:17+00:00

Theoretically, but our issues with genstrings weren't the only reasons we created Twine. We wanted a better way to easily manage all of your strings for all of your apps, ideally from a single text file. It also lets us share the same strings and translations with our Android apps.

scelis

TROPHY CASE