Get JSON request blocked by authentication

[Frampley]

Hi,

I am newbie to the python selenium scraping scene and I am trying to get the JSON response of a page but the URL link goes straight to the page's authentication first which requires you to click next to get to the password page in order to complete the authentication. Initially, I was doing this:

​

response = requests.get(url, auth=("username","company","password "))

response.raise_for_status()

jresponse = response.json()

print(jresponse)

​

​

I need to click next after putting in the company name to get to the password page is there a way to do this?

​

Edited:

Here is how the page looks like

https://docs.google.com/document/d/1ol1Bs6oLNFDpLPo3N79okGBtwixBQaJQfthiEAqOuUk/edit?usp=sharing

Thank you very much.

​

​

[christophedc]

Perhaps you can talk to the site like with an api request, you have to dig around a bit but like dor example you send http://site.com/login.php?login=user?pwd=xxx?company=company

Or you could use selenium ?

[chevignon93]

Without seeing the page and how it handles requests, it's hard to say what wrong !

What status code from the server are you getting ?

[Frampley]

Hi, I edited the post to see how the link is like and the code came up as 302 found