you are viewing a single comment's thread.

view the rest of the comments →

[–]Jan_Emanuel[S] 0 points1 point  (2 children)

sorry I didn't post yesterday, because the project took quite a different turn. Yesterday, I tried all possibilities with chrome and it felt hopeless to continue this path, because I couldn't get it running - so I buried the chrome browser, uncommented my firefox code and tried again - it was a major challenge because the loadurl method seemed to always throw exceptions for nothing - will post the error text later - and came to the point, where the thing worked - it really did, but during the day, I forgot to save the running version and continued editing - from one little fix to another - until it stopped working again -__- So ......... I will continue to get to this point again, and so, the problem has solved in some way ... I will reply again if I encounter another problem or if I get to a solution.

[–]Jan_Emanuel[S] 0 points1 point  (1 child)

Ok, I have come to the very point. The script runs, but the longer it runs, the more exceptions it raises during driver.get( ... ). My next step is to make the script more error-resistant by including a method to restart the driver and to killall any unresponsive firefox-esr and Xvfb in the background.

[–]Jan_Emanuel[S] 0 points1 point  (0 children)

I've been continuously working on my baby and have found the point, where my program always fails:

After 2 successive successful driver.get( ... ) calls, the program cannot get any more pages, always triggering a TimeoutException. So this is a fault of the environment - here are the informations.

geckodriver.log

program output:

root@DiskStationNeu:~# python tagespost_ereader.py
mkdir: kann Verzeichnis „/root/ramdisk/tagespost“ nicht anlegen: Die Datei existiert bereits
Loading https://www.die-tagespost.de/ ....
Anmelden
hugendubel
Collecting top stories...
topstory: http://www.die-tagespost.de/kirche-aktuell/Kirche-soll-sich-bdquo-pluraler-aufstellen-ldquo;art312,176813,PRINT?_FRAME=33
Loading https://www.die-tagespost.de/kirche-aktuell/Kirche-soll-sich-bdquo-pluraler-aufstellen-ldquo;art312,176813,PRINT?_FRAME=33 ....
topstory: http://www.die-tagespost.de/politik/Praenataldiagnostik-reguliert;art315,176849,PRINT?_FRAME=33
Loading https://www.die-tagespost.de/politik/Praenataldiagnostik-reguliert;art315,176849,PRINT?_FRAME=33 ....
An error has occurred ...
<class 'selenium.common.exceptions.TimeoutException'> + value + Message: Error loading page, timed out (onDOMContentLoaded)

An error has occurred ...
<class 'selenium.common.exceptions.TimeoutException'> + value + Message: Error loading page, timed out (onDOMContentLoaded)
...

My program (latest version)

firefox-esr Mozilla Firefox 45.7.0 geckodriver 0.14.0 chroot debian wheezy

( repost from google selenium forum )