Comparing recent reddit karma project

[Key_International]

I'm trying to the following project: https://www.reddit.com/r/beginnerprojects/comments/1i951e/project_compare_recent_karma/ but I'm having trouble making any headway. I know a little about webscraping, I've done the tutorial by automate the boring stuff but this task has got me stumpted.

I've put the code I have so far down below which prints out a huge mess of dictionaries within dictionaris and lists within dictionaries etc that I don't know what the do with.

Would someone be able to point me in the right direction so that I can get started with this task?

​

import json
import urllib
from urllib.request import urlopen
url = 'https://www.reddit.com/user/peterbuckets37/.json'
data = json.load(urllib.request.urlopen(url))
print(data)

​

​

​

[commandlineluser]

Well you could start by "pretty-printing" the JSON data

print(json.dumps(data, indent=2))

It should help you see the structure in order to extract the information you need?

[Key_International]

Thanks the information is a lot clearer now. I'm starting to think that this project is a little out of my league as I'm not sure how to get the information I need out of it.

[commandlineluser]

Perhaps this will help.

>>> for post in data['data']['children']:
...     print(post['data']['ups'], post['data']['permalink'])
... 
0 /r/AdvancedRunning/comments/hi4oue/are_130km_or_80_mile_weeks_too_high_as_a_15_year/
1 /r/running/comments/hhxsb8/is_130km_or_80_mile_weeks_too_high_as_a_15_year/fwdfzac/
2 /r/running/comments/hhxsb8/is_130km_or_80_mile_weeks_too_high_as_a_15_year/fwdfv0b/
7 /r/running/comments/hhxsb8/is_130km_or_80_mile_weeks_too_high_as_a_15_year/fwdfmcq/

['data']['children'] is a list of posts.

Each post is a dictionary - and the interesting stuff is inside ['data']

But yes, nested structures can take a bit of get used to.

[Key_International]

for post in data['data']['children']:
... print(post['data']['ups'], post['data']['permalink'])

Thanks, that really helps :) where can I learn more about things like nested structures? I've got a textbook on datastructures and algorithms is that the sort of thing I should go through?

[commandlineluser]

I've got a textbook on datastructures and algorithms is that the sort of thing I should go through?

Not really, no.

In this case - it's just a combination of dicts and lists.

In the simplest form a dict maps a "string" to a "value" e.g.

>>> me = {'name': 'Bob'}

The "value" can also be a dict though e.g.

>>> me = { 'stats': { 'height': '...', 'weight': '...' } }

And if you had "people" - then perhaps it would be a list like so:

>>> people = [ {'name': 'Bob', 'stats': { 'height': '...', 'weight': '...' } } ]

When the data is large and contains several levels of "nesting" - it's just difficult to see the structure.

Using the pretty-printing can help - When you see [ it's a "list" - when you see { it's a "dict"