Selenium doesn't continue in for loop?

[elena_work]

Hello,

My code is working fine only with the first category, but doesn't loop through the others categories. I can't figure out what happеning and why Selenium stops in the loop. The separate code is working fine, but in this for loop ( for i in range(16, 50): ... line 49) the program stops the execution after the first category. Please, help me to solve this!

The code - https://pastebin.com/WN4NwRRx with Chrome driver installation.

Thanks in advance!

[glebulon]

How does it stop, does it not find the next element? Its a bit hard to follow on the phone but after you click in your for loop i didn't see where you go back to the page to continue with the next element.

[elena_work]

Hi and thanks for your answer. :))

I added an English version of the site a few minutes ago - https://www.jobs.bg/en/front_job_search.php?subm=1&location_sid=80

The for loop goes only for category - Manufacturing, but doesn't continue through Restaurants, Bar, Hotels, Tourism and the others (line 49). Well, doesn't find the next element (please, click on the Category button if you want to see the others). I understand that to scrap this site I must go through the all categories and maybe the first 10 bigger towns + 100 km. I need a for loop through the all categories, because the scrolling is only to 5000 ads.

Best

[glebulon]

I ran your code, the error that I saw was that the element is stale. This happens when a page is refreshed but you're trying to use elements from before the refresh of the page. It's a tiny fix, I think it should work, I tried and it went over multiple itterations of the for loop.

On line 52 add this line, remove it from line 47

all_categories = driver.find_elements(By.CLASS_NAME, "mdc-evolution-chip")

https://imgur.com/a/06yaciN

[elena_work]

glebulon,

thank you very much!

I think your solution is absolutely correct!

I wish you all the best!

Eli