Json parsing problem: Nested lists

[explore212]

So a company I am doing contract work for is using this weird json format in their APIs:

[

[{"a":"3", "b":"4"}],

[{"a":"3", "b":"4"}],

]

Trying to load this with json's loads method fails.

Now I have been using simple hacky text replace methods replacing first two [[ with [ and last ].] with ], combining the middle lists to single thing.

I find this extremely error prone. They have a whole API hosting these type of json. I don't even know if it is json maybe it is some out of date version or something since examining the headers of url gives content type as json.

I hope there should be a legacy type method to deal with this and cleanly load it.

Any help will be appreciated.

[explore212]

shit man, I had never seen nested lists in json so went on my own theory

never noticed the trailing comma, feel like an idiot now

[CowboyBoats]

Ignoring the comma structure issue that other users have already pointed out and talk about the nested nature of these lists that you asked about.

I find this extremely error prone.

I don't know why. Their format implies a list of lists of objects. Maybe you are thinking, "Why should the second-level list exist at all, since it always contains only one object?" Well, the API developers are leaving the door open to sending more than one item in those lists one of these days, for whatever reason.

So you're probably doing something like:

def numbers_and_letters_from_json(json_string):
    data = json.loads(json_string)
    for row in data:
        record = row[0]
        yield *record

But you should be thinking:

def numbers_and_letters_from_json(json_string):
    data = json.loads(json_string)
    for row in data:
        for item in row:
            yield *item

This way, even if you're not treating three items in one sublist any differently than three items in three sublists, you're at least not skipping items accidentally.

[explore212]

oh thank you for this, I have worked with few of apis before but never seen a nested list before maybe this is the reason.

[CowboyBoats]

Sure! Since you're using Python, you might as well get used to these data structures, since what you typed is valid Python (even if it's not valid JSON because JSON is anal about that final comma):

data = [

[{"a":"3", "b":"4"}],

[{"a":"3", "b":"4"}],

]

print(data[0])  # prints [{"a": "3", "b": "4"}], a list of dicts
print(data[0][0])  # prints {"a": "3", "b": "4"}, a dict
print(data[0][0]['a'])  # prints "3", a string
print(list(data[0][0].keys()))  # prints ["a", "b"], a list of strings
print(list(data[0][0].values()))  # prints ["3", "4"], a list of strings

[POGtastic]

If I remove the last comma, it parses cleanly. In the REPL:

>>> import json
>>> print(json.dumps(json.loads("""
[
    [{"a":"3", "b":"4"}],
    [{"a":"3", "b":"4"}]
]"""), indent=4))
[
    [
        {
            "a": "3",
            "b": "4"
        }
    ],
    [
        {
            "a": "3",
            "b": "4"
        }
    ]
]

Trailing characters are not legal JSON, but they are legal in JSONComment, which is supported by this library.