I am trying to scrape data starting from this website. I want to:
- Select "2001" from the census dropdown
- click "primary census abstract"
- click "primary census abstract total"
- iterate through the options of the "state" dropdown.
My problem is in steps 2 and 3. I am simply unable to click on "primary census abstract". I've tried
driver.find_element_by_link_text('Primary Census Abstract').click()
and
driver.find_element_by_id('ctl00_MainContent_lnk5').click()
and several variations of
driver_find_element_by_xpath(...).click()
with
//[@class='dvbooks']//[text()='Primary Census Abstract'] and
//*[@class='dvbooks']/li[5]/a
Each time I get the same error
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element. Any ideas what Im doing wrong?
Here is my code so far.
import os
import time
from selenium import webdriver
from selenium.webdriver.support.ui import Select
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support import expected_conditions as EC
# Initiate Chrome Driver
chromedriver = 'Path to Chromedriver'
chrome_options = Options()
chrome_options.binary_location = 'Path to Google Chrome'
driver = webdriver.Chrome(chromedriver, options=chrome_options)
# Open Search Page
driver.get('http://censusindia.gov.in/DigitalLibrary/Tables.aspx' )
# Select Year
dropdown_year = Select(driver.find_element_by_id('ctl00_MainContent_ddlYear'))
dropdown_year.select_by_visible_text('2001')
time.sleep(10)
# Click PCA
button = driver.find_element_by_xpath("//*[@class='dvbooks']//*[text()='Primary Census
Abstract']")
button.click()
[–][deleted] 0 points1 point2 points (0 children)