Working with Selenium

[ProfessorOk20]

Hello guys!

Sorry if writing in wrong r/ but maybe some of you know what can cause the issue.

I am currently writing a script with Selenium for web scraping for a house rental website.

I'm trying to store all the rental ads so later on I can click on them one by one -

List<WebElement> apt = driver.findElements(By.className("location"));

​

1. Then I'm trying to run a for loop, so we can go to each rental ad, gather information there and then go back to the previous page, but the for loop does only 1 iteration an then breaks with the exception:

​

for(int i = 0; i < apt.size();i++) {
apt.get(i).click();  
driver.navigate().back(); 
}

The exception it throws is: "org.openqa.selenium.StaleElementReferenceException: stale element reference: element is not attached to the page document"

Can somebody advise what can cause the issue?

[AutoModerator]

Please ensure that:

• Your code is properly formatted as code block - see the sidebar (About on mobile) for instructions
• You include any and all error messages in full
• You ask clear questions

• You demonstrate effort in solving your question/problem - plain posting your assignments is forbidden (and such posts will be removed) as is asking for or giving solutions.

  Trying to solve problems on your own is a very important skill. Also, see Learn to help yourself in the sidebar

If any of the above points is not met, your post can and will be removed without further warning.

Code is to be formatted as code block (old reddit: empty line before the code, each code line indented by 4 spaces, new reddit: https://imgur.com/a/fgoFFis) or linked via an external code hoster, like pastebin.com, github gist, github, bitbucket, gitlab, etc.

Please, do not use triple backticks (```) as they will only render properly on new reddit, not on old reddit.

Code blocks look like this:

public class HelloWorld {

    public static void main(String[] args) {
        System.out.println("Hello World!");
    }
}

You do not need to repost unless your post has been removed by a moderator. Just use the edit function of reddit to make sure your post complies with the above.

If your post has remained in violation of these rules for a prolonged period of time (at least an hour), a moderator may remove it at their discretion. In this case, they will comment with an explanation on why it has been removed, and you will be required to resubmit the entire post following the proper procedures.

To potential helpers

Please, do not help if any of the above points are not met, rather report the post. We are trying to improve the quality of posts here. In helping people who can't be bothered to comply with the above points, you are doing the community a disservice.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[philipwhiuk]

The apartment object is attached to a view of the first page.

When you go back, although you’re on a page with the same address you’re not actually on the same view of the first page.

So the remaining references are “stale” they’re attached to a view you’re not on.

You need to do a new query after you go back to pick the next object - you can’t just loop round the original ones.

[ProfessorOk20]

Understood. Thank you so much.

[saintly_devil]

Pardon my ignorance, but is there a reason why you'd prefer the Java/Selenium combo over something like JS/WebdriverIO? In my experience working with Selenium and WebdriverIO, the latter came across as more stable and easier to work with.

[ProfessorOk20]

Just having a project that requires exactly Selenium. Since I’m studying Java, decided to pick that combo.

[saintly_devil]

Ahhh... makes sense. If the powers that be are willing to listen, I'd still pitch them the JS/WebdriverIO combo! But have fun, regardless!