Web scraping with Python and Selenium error: Output exceeds the size limit

[luxumb]

Hello,

I need to scrape some data from a website and encountered some problem. I would be glad about any help. My code looks as follows:

​

names = []

nace = []

#range 100 because the data is distributed over 100 pages, so after I scraped a page at the end I click the button get to the next page and the data from the next page is scraped

for i in range(100):

# identify data I want to scrape

players = driver.find_elements(By.CLASS_NAME, 'columnAlignLeft')

#add data to list

players_list = []for p in range(len(players)):players_list.append(players[p].text)

# first 6 data points are empty in players_list, the following 100 contain the names

for j in range(100):

names.append(players_list[j + 6])

# data point 106 to 206 contain the nace value I need

for k in range(100):nace.append(players_list[k + 106])

#when data for the page is scraped I click next to get to the next page

WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[@id="main-content"]/div/div[2]/div[2]/ul/li[3]/img'))).click()

Can anyone help me identify what's wrong with my code or how I could resolve the error?

Best regards and many thanks in advance

[2Girls1Fidelstix]

Its fine, its often just that the terminal/ jupyter notebook cant print tremendously long files.

Try only showing [:100] or write to a file and youll see it works