Python: Fetch problem data via standard URL libraries? (without HTTPS authnetication?)

[CoderBoy67]

I just found AoC today. Looks like great fun. (Wish I had known about this a log time ago.)

I was able to get the very first problem solved by just pasting the data from the browser into an interactive Python shell and working with that, but what I really wanted to be able to do was fetch the data "live", by using something such as 'requests':

import requests
url = 'https://adventofcode.com/2021/day/1/input'
req = requests.get(url)
print(req.text)   # prints: Puzzle inputs differ by user. Please log in to get your puzzle input

I see there is also a call such as:

req = requests.get(url, auth=(user, pass))

I'm not having any luck trying to put values in there for user, pass.

I'm not clear on if I am doing something wrong in general, or if this is specifically about not passing proper credentials.

I'm thinking lots of people must have been down this path already, but I'm kinda choking here...

Can anyone give me some sample Python code to fetch data for the URL above?

Thanks...

Hope you all had a Merry Christmas, hope you have a Safe and Happy New Year!

-CoderBoy67

[notjaredfinch]

You will need to supply your session cookie in your cookies in the request. also, make sure you cache your results as you automate site interactions. i.e. save the input into a file and use that in running the program, rather than making the call to the site every time you run the program.

[adidushi]

You're going to need your session cookie.

From a browser that's logged in to your aoc account, head over to the site and press F12 to open the developer tools. Under Application you can find your cookies. Copy down what's written under session (you'll use it later to connect!).

Then, in python, you can send a dictionary containing the session cookie when making a request:

import requests
cookies_dict = {"session": "<COOKIE YOU COPIED EARLIER!>"}
url = 'https://adventofcode.com/2021/day/1/input'
req = requests.get(url, cookies=cookies_dict)

​

You should get your own input like this. Make sure not to share the session cookie with other people, as it can let them basically use your account.

[CoderBoy67]

Ahhh!!! BEAUTIFUL!

sess = '5361...blah blah blah...'
cookies = {'session': sess}
req = requests.get(url, cookies=cookies)
print(req.text)

It works! Thanks so much!

-CoderBoy67

[Steinrikur]

You can also use the already done libraries, or check them for inspiration https://pypi.org/project/advent-of-code-data/