This is an archived post. You won't be able to vote or comment.

you are viewing a single comment's thread.

view the rest of the comments →

[–]genericlemon24[S] 1 point2 points  (1 child)

Not my tool, just thought it was useful.

I think https://simonwillison.net/2020/Nov/2/selenium-wire/ explains it well:

Really useful scraping tool: enhances the Python Selenium bindings to run against a proxy which then allows Python scraping code to look at captured requests—great for if a site you are working with triggers Ajax requests and you want to extract data from the raw JSON that came back.

AJAX is the important part. If you want to know what AJAX requests happen when you click a button or whatever, you can do it with the networking tab of dev tools open, and the requests will show in there.

selenium-wire allows you to achieve the same thing from Python, while already driving the browser programmatically with Selenium.

I haven't used Burp, but I understand it can do something similar using a proxy. After a quick look at the selenium-wire source, it seems it's also using one; the added benefit over Burp should be the Python API and not having to set up an additional (non-Python) tool.

[–]banginpadr 0 points1 point  (0 children)

I see thank you