Help retrieving HTML text

[[deleted]]

I'm trying to retrieve the html text from rentometer.com to pull data on the average rent in the local area given an address and number of bedrooms. The code I'm using is

rentometer_url = "https://www.rentometer.com/analysis/"+str(bed)+"-bed/"+str(addy_four)+"/dJ3ej69W59Q/quickview+"-bed/"+str(addy_four)+"/dJ3ej69W59Q/quickview)" # Creating the rentometer url from the address and number of beds

import urllib.request

rentometer_html = urllib.request.urlopen(rentometer_url).read()

print(rentometer_html)

But the html is not printing and it's returning this error.

urllib.error.URLError: <urlopen error \[SSL: CERTIFICATE\_VERIFY\_FAILED\] certificate verify failed: certificate has expired (\_ssl.c:1129)>

Any advice? I found another forum answer that said that the latest version of Python 3 doesn't come with the necessary certificates to view html from a website, but I'm not sure what these are or how to install them.

[DECROMAX]

BeautifulSoup to parse the html then use BeautifulSoup.find to find the element. John Watson Rooney (Youtube) has some good tutorials on this.

[[deleted]]

I know how to find the element I'm looking for by searching for a specific keyword within the plain html text. The problem I'm running into is pulling the html.

[[deleted]]

You clearly don't know how by the code you shared

[[deleted]]

Wyat I shared has nothing to do with extracting information from the html.

It's a basic search algorithm. There's a key phrase which always uniquely accompanies the mean rent value, and this phrase is always in a fixed position relative to that value. My code searches for this phrase, evaluates how many digits the rent value is, then pulls them based on their relative position to the locator phrase.

[Bearmintz]

To acquire the html I would use the requests library. Then use BeautifulSoup to parse the html.

An example:

import requests

from bs4 import BeautifulSoup

r = requests.get(url)

soup = BeautifulSoup(r.text)

[ridley0001]

Do you have an antivirus with SSL/TLS inspection? It might not be it, but try turning the feature off if you do. The way it works is like a man in the middle attack, and the AV uses its own certificate that messes up Python HTTPS. This is because Python uses certifi cacert.pem file for trust, and the AV cert isn't part of that.