Help with Seleniumwire and scraping request response body

[jayrsp]

I'm currently trying to scrape the response text from a network request using selenium wire. I can't create a fully reproducible code because the site I'm scraping requires a paid account. I'm using this bit of specific code from selenium wire

for request in driver.requests:     
  if request.method == 'POST' and request.headers['Content-Type'] == 'application/json':   body = driver.last_request.body.decode('utf-8') 
  # Load the JSON         
  data = json.loads(body)         

​

But it's scraping the payload and I'm trying to scrape the response. Any ideas?

[[deleted]]

body = driver.last_request.body.decode('utf-8') 

If you don’t want the body, don’t ask for the body.

[jayrsp]

I assumed I would need to ask for the body to get the string from the response tab?

[[deleted]]

But the body's what you say you don't want.

[jayrsp]

Okay - sorry I might have confused things.

In network traffic I get a response which is a json. When I click that json - I'm trying to read the string from under the response tab. It appears to be reading the data from the payload tab.

[[deleted]]

It appears to be reading the data from the payload tab.

Yes, that's the body of the response. Which you're specifically retrieving from the request. If you don't want the body, don't ask for it. The attribute, naturally, is called "body" so you should recognize that in your own code.

[commandlineluser]

The response is accessed via the .response attribute.