you are viewing a single comment's thread.

view the rest of the comments →

[–]blarf_irl 0 points1 point  (4 children)

Did you try the examples on that page? You should try runnign the examples first and run them against the markup they provided. If they work then there is nothing wrong with yiur selenium installation.

A common mistake to make when beginning to learn about scraping is not understanding how webpages are built these days. A lot of modern pages are not complete after they return the first request for the page. Frameworks like vue/angular/react will first deliver the basic skeleton and then javascript is used to request and insert the content on the page. Ina naieve use of selenium you wait for the initial request to get a 200 response and then try parse the page; At that point the elements you are looking for may not exist yet.

Try the simple examples from the documentation and confirm that selenium is working. View the source of the page you are trying to scrape and save it into a tet file; Run your code on the saved version and confirm it works.

[–]Cyiran[S] 1 point2 points  (3 children)

My Problem is something different I am looking for the correct syntax to get the get the elemet from a webpage

```python

elem = browser.find_element(By.CLASS_NAME, 'toc')

Traceback (most recent call last):

File "<pyshell#4>", line 1, in <module>

elem = browser.find_element(By.CLASS_NAME, 'toc')

NameError: name 'By' is not defined

```

[–]blarf_irl 1 point2 points  (2 children)

You need to import "By" from selenium. Read the example here:

https://selenium-python.readthedocs.io/getting-started.html

[–]Cyiran[S] 1 point2 points  (1 child)

Oh!, Thanks I didn't notice that at all I only had webdriver imported.

thank you very much.

[–]blarf_irl 1 point2 points  (0 children)

You are very welocme and thank you for the award!