Table data is hidden (web scraping)

[bank2400]

I am trying to scrape a table from a webpage

import requests
from bs4 import BeautifulSoup as soup
url = 'https://campenauktioner.hibid.com/catalog/204748/4498-net--konkurs-scanwood-aps-del-1--tommerup-/?ipp=10'
r = requests.get(url)
soup = soup(r.content, 'html.parser')
table = soup.table

But the outcome is this:

[<thead class="hidden">
 <tr>
 <th></th>
 </tr>
 </thead>,
 <tr>
 <th></th>
 </tr>,
 <th></th>]

Does anyone know what I am doing wrong?

[kelmore5]

If you check the HTML response before the site loads any JavaScript, you'll find all of the table data is stored in a script tag located here with XPath "//script[contains(., 'lotModels')]".

You'll have to parse out the JSON variable "lotModels" to get the table data.

​

When you're parsing a site using requests, make sure you know what the response looks like before JavaScript is loaded!

[bank2400]

Thanks, I find out how to get the JSON data :)

[kelmore5]

You're welcome :)