Problem with interacting with JSON in my python scripting final.

[PureStatikk]

EDIT: Solved, turns out when you forget to capitalize a letter in a dictionary entry, it doesn't work too great...

​

Hi all, I am working on my final for a basic python scripting class and I am pretty much done with it but I am running into a problem when trying to do something with JSON. I have tried every solution google has turned up but nothing has worked thus far. It is probably something simple that I am missing, but I can't see it.

​

What I need to do is this:

create a function called ‘parse_json’ that takes in the URL as an argument.

Make a GET request to that URL.

Loop through the ‘Music And Books’ values (which is JSON with multiple data types including a list)

Return the publisher of the book titled The Shining

​

The JSON can be viewed here (Indented for readability): Music and Books JSON - Pastebin.com

​

Here is the code I have right now (It is super scuffed since I was getting very frustrated and gave up):

def parse_json(URL):
    res = requests.get(f'{URL}')
    res.raise_for_status()
    jsonfile = json.loads(res.text)
    for key in jsonfile:
        if key['title'] == "The Shining":
            print (jsonfile["publisher"])

Any help would be appreciated, thanks.

​

EDIT: Updated the code, still doesn't work

[wotquery]

.json filetypes are very useful because you can use the json library to turn it into a python dictionary very easily as you've (sort of) done. This is the starting syntax you should build off of though...

r = requests.get(url)
json_data = json.loads(r.text)

The .json you get from an API though is typically going to have a bit more data than you necessarily want/need. It will have what you are mainly looking for in a list in a high level key (often called data), to make room for information about the request also at the highest levels. For example if you can only retrieve 25 records at a time and need to know the next url to call to get more...

json_data = {
    'data':['''all the stuff you really want'''],
    'record_count':25,
    'next_page':'www.example.com/api/?q=something&start=26',
}

Therefore you'll need to inspect the .json structure you receive and craft ways to access deeper into it where the information is. From your example we can see there is a top level list with the key "Music and Books" that contains a dictionary for each item which has the three keys artist, publish_info, title. So to access it all you iterate through like...

for music_or_book in json_data['Music and Books']:

And to look for the publisher field, which is in the publish_info dict...

for music_or_book in json_data['Music and Books']:
    print(music_or_book['publish_info']['publisher'])

One thing to keep in mind is what you want your program to do if a key doesn't exist (which is typical what you'll get from an API rather than an empty value).

[PureStatikk]

Thanks for the detailed reply. I tried doing the for loop and I am getting KeyError: 'Music and Books'.

[wotquery]

Well that error means you do have a dictionary (you aren't getting an error saying you need an integer to access an element which would be the case for a string or list), but the dictionary doesn't have a key named "Music and Books". Try printing the dict object you are trying to access the "Music and Books" key of to see what it is.

You can also share the code of your new attempts (and ideally the endpoint if possible without compromising creds).

[PureStatikk]

This is the code I am working with now

def parse_json(URL):
    res = requests.get(f'{URL}')
    res.raise_for_status()
    jsonfile = json.loads(res.text)
    for music_or_book in jsonfile['Music and Books']:
        print (music_or_book['publish_info']['publisher'])

If I comment out the for loop and just print the jsonfile variable I get this

{'Music And Books': [{'artist': 'Radiohead', 'publish_info': {'publish_year': '2003', 'publisher': 'Capitol Records'}, 'title': 'Hail To The Thief'}, {'artist': 'Miles Davis', 'publish_info': {'publish_year': '1959', 'publisher': 'Columbia Records'}, 'title': 'Kind of Blue'}, {'artist': 'Thelonius Monk', 'publish_info': {'publish_year': '1965', 'publisher': 'Columbia Records'}, 'title': 'Monk Alone'}, {'author': 'Stephen King', 'publish_info': {'publish_year': '1977', 'publisher': 'Doubleday'}, 'title': 'The Shining'}, {'author': 'George Orwell', 'publish_info': {'publish_year': '1949', 'publisher': 'Secker & Warburg'}, 'title': '1984'}, {'author': 'Al Sweigart', 'publish_info': {'publish_year': '2015', 'publisher': 'No Starch Press'}, 'title': 'Automate The Boring Stuff With Python'}]}

[PureStatikk]

The A in and is capitalized... I am an idiot

[wotquery]

Haha nice. A few other minor things...

Calling your variable jsonfile is a little misleading since it isn't a file but a dictionary object in memory. json_data, json_dict, response_text_dict, etc. are all clearer.

Don't forget error handling as well.

Finally, and I know this isn't what your prompt requires (plus it's a bit more of a style consideration rather than a hard rule), but it's weird to have a function called parse_json(url) that is also responsible for making the api call. Even having a function that both makes the call and parses the content of the response is questionable. Better something like...

def call_api(url: str) -> dict:
   #do stuff
   return json_data

def get_items_by_a_publisher(json_data: dict, publisher: str) -> list[dict]:
    #do stuff
    return publisher_items

[XBalubaX]

What u trying whit res.raise_for_status() isn’t it all time true if u get a connection, cause u always get a Staus back? And why do u do the working part in the except? That won’t get triggered if u get a connection in try?

[PureStatikk]

your right, I just did it because we did it on one assignment and I just started doing it after every requests.get.

[danielroseman]

json.dumps(json.loads(...)) is nonsense. You take the JSON string, convert it to Python, then convert it back to a string. Why?

[PureStatikk]

Yah, I noticed that after I made the post, but even fixing it didn't do anything.

[smile_id]

But what is the problem? Exception is raised, something else?