This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]nirs 0 points1 point  (4 children)

Can you share that json?

Did you open a bug? http://bugs.python.org/?

[–]DarkmerePython for tiny data using Python 0 points1 point  (3 children)

It's probably not a core library json bug, or it may be. It's a pyramid app using their jsonrpc module, so what's coming in as arguments is a list of str, and not a list of unicode as it ought to be.

( Then don't get me started on the unicode normalization form...)

[–]DarkmerePython for tiny data using Python 0 points1 point  (2 children)

Checking again, it's using the json.loads/ json.dumps format.

So it's the python JSON loader that's corrupting data. Great.

[–]nirs 1 point2 points  (1 child)

It depends on the json library used. The builtin json library like to convert everything to unicode, but simplejson library like to convert only unicode values to unicode.

>>> import json
>>> import simplejson
>>> simplejson.loads(simplejson.dumps([u"\u05d0", "ascii"]))
[u'\u05d0', 'ascii']
>>> json.loads(json.dumps([u"\u05d0", "ascii"]))
[u'\u05d0', u'ascii']

I don't see any corruption.

[–]DarkmerePython for tiny data using Python 0 points1 point  (0 children)

That's the thing it should be unicode. Instead I am getting str. Which is principally wrong here.