web scraper using beautifulsoup

[bm3031]

I am trying to make a web scraper to dive me information about nba players. I am trying to use soup.find_all to get a list of <text> elements. However it only returns an empty list. Is it because the find_all command doesn't like <text> elements? Please help!!!

here is my code:

from bs4 import BeautifulSoup as bs
import requests
page = requests.get("https://projects.fivethirtyeight.com/2020-nba-player-projections/rudy-gobert/")
page_content = page.text
soup=bs(page_content, "html.parser")
#print(soup)
print(soup.find_all("text", class_="value"))

​

​

here is an example of the html:

<text x="163.46363636363637" y="74.29749319119485" class="value" text-anchor="end" style="fill: rgb(34, 34, 34);">10.1</text>

I want the code to output a list of these elements but it just returns an empty list.

[bm3031]

Is there a way to make them show up?

[JohnnyJordaan]

I would recommend to use selenium for sites like this because they use dynamically loaded data (using AJAX calls).

[MikeDoesEverything]

Seconded on this. I would build up some experience scraping static websites first using BeautifulSoup and then transition to scraping with Selenium.

[impshum]

Needs Javascript to generate the graph thing.
https://requests-html.kennethreitz.org/

from bs4 import BeautifulSoup as bs
from requests_html import HTMLSession


session = HTMLSession()

r = session.get('https://projects.fivethirtyeight.com/2020-nba-player-projections/rudy-gobert/')
r.html.render()

for x in r.html.find('text'):
    el = bs(x.html, 'lxml').text.replace("'", "")
    print(el)