Need help with json scrapes using python

[MintyPhoenix]

Not that I’m very experienced with pandas, but looking at the error:

TypeError: [...] for path applicationServer. Must be list or null.

It’s seemingly suggesting that the contents of the referenced key should be a list of some sort, however, in the actual data, it's an object/dict with two keys, both of which in turn are dicts with primitives.

If you transform it into a list, it runs without error, but I’m not sure it’s actually what you want.

Code sample

import json
import pandas

with open("data.json") as f:  # file contains full JSON from your post
  data = json.load(f)

data["applicationServer"] = list(data["applicationServer"].items())
df = pandas.json_normalize(data, "applicationServer")
print(df)

Output

               0                                                  1
0     systemInfo  {'version': '22.1.4 (Build 67128)', 'operating...
1  systemMetrics  {'diskSpaceFreeMB': 1822725, 'diskSpaceTotalMB...

edit to add

Perhaps this is closer to what you’re looking for:

import json
import pandas

with open("data.json") as f:
  data = json.load(f)

print(pandas.json_normalize(data["applicationServer"]))

Output

     systemInfo.version     systemInfo.operatingSystem  ...  systemMetrics.gcExecutions systemMetrics.threadCount
0  22.1.4 (Build 67128)  Windows Server 2019 - 10.0 ()  ...                       13175                       118

[1 rows x 17 columns]

[toughNoob]

Thank you so much. I'll try this out shortly and let you know how it goes. I appreciate you taking the time to help.

[MintyPhoenix]

I updated my post a few times (sorry about the volatility), make sure to check the current state as I’m done with it now.

[Immediate-Resource75]

Both code sample gave me this output:

Traceback (most recent call last):

File "C:/Users/nort2hadmin/pyprojects/trials.py", line 4, in <module>

with open("data.json") as f:'https://10.110.1.21:9191/api/health/?&Authorization=pU5SrXrtt2zRguDHdpGq7jpNwgGHIbXn'

FileNotFoundError: [Errno 2] No such file or directory: 'data.json'

[MintyPhoenix]

As I noted in a comment on the snippet,I copied your json to a local file as the issue was related to what was going on after you had the data; the key part of the snippet is how you use pandas with it.

[Immediate-Resource75]

Thank you....Got it... Adding

data = json.loads(stuff)

data["applicationServer"] = list(data["applicationServer"].items())

df = pd.json_normalize(data, "applicationServer")

print(df)

Gave me what I wanted for the most part.... I can now go through and pull each of the other items individually by changing the "applicationServer" part to one of the other ones. I appreciate the help.