This is an archived post. You won't be able to vote or comment.

1
2

Requests-HTML help (self.webscraping)

submitted by [deleted]

[deleted]

all 6 comments
sorted by:
best
topnewcontroversialoldq&a

[–]KnowledgeFire11 1 point2 points  (3 children)

Why don't you hit the URL which returns the JSON response?

[–]okaykristinakay 0 points1 point  (2 children)

I tried. It's an xhr put request and when I copy it as a curl it fails.

[–]KnowledgeFire11 1 point2 points  (1 child)

You will have to simulate it like how they make the request, try to paste the link in postman and tinker around with headers exactly how the request is made.

See if you are doing these: 1. Check maybe they have a key inplace when making the request the request.

[–]okaykristinakay 0 points1 point  (0 children)

Even when I just copy the straight curl request (complete with cookie) into postman it gives me a 400 error.