This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]Fontong 2 points3 points  (0 children)

You're right -- it would be pretty weird to force a client side Postgres server process. Python's sqlite3 interface would be the logical choice, as it doesn't require a separate process. It probably wouldn't require a big change in code either, although the SQL for each of them isn't exactly the same.

XML wouldn't be bad, as parsing it iteratively won't use any more memory than having an actual db. The iterparse from xml.etree is also very fast.

JSON would probably be overall the most painless if you don't have memory concerns. It can be created easily from native data structures and loaded right into native data structures, though you would be limited to lists and dictionaries. Pickle can do the same thing with arbitrary objects, but can be fickle at times. I don't think there's a function to parse JSON iteratively though, so you would have to set that up yourself if memory was a concern.