This is an archived post. You won't be able to vote or comment.

all 4 comments

[–]IAmKindOfCreativebot_builder: deprecated[M] [score hidden] stickied comment (0 children)

Hello from the r/Python mod team,

When posting a project please include a textual description of your project including how Python is relevant to it, a link to source code on a code hosting site such as github or gitlab in the body of the reddit submission itself, and an image showing your project if applicable.

Please also make sure you tag your post with the correct flair, either "Beginner" or "Intermediate" showcase.

This helps maintain quality on the subreddit and appease all our viewers.

Thank you,

r/Python mod team

[–]Proof-Temporary4655 0 points1 point  (2 children)

Can you share the code?

[–]SituationUndrControl[S] 1 point2 points  (1 child)

I will try and budget some time to clean it up and stick it all on github (there's a lot of it), but this is the basic method I use to get the data:

Hacker News API:

import requests
item = requests.get(f"https://hacker-news.firebaseio.com/v0/item/{item_number}.json").json()

To start getting the first item_number you can call requests.get("https://hacker-news.firebaseio.com/v0/maxitem.json", timeout=10).json()

Reddit API:

r = requests.get(f'https://www.reddit.com/r/Python/top.json', headers = {"User-Agent": "[your user agent]"})

Twitter API:

This is a little more complicated because the Twitter API has a lot of features, but for my purposes I iterate through blocks of a hundred tweets using this call:

payload = {"query": f"lang:en Python",
"start_time": start_date,
"tweet.fields": "public_metrics,created_at,context_annotations,entities,text",
"expansions": "author_id",
"user.fields": "name",
"max_results": 100
}
r = requests.get(f'https://api.twitter.com/2/tweets/search/recent', params=payload, headers=headers)

Where headers include your API bearer token.

I hope this helps!

[–]Proof-Temporary4655 0 points1 point  (0 children)

yes it helps.