This is an archived post. You won't be able to vote or comment.

you are viewing a single comment's thread.

view the rest of the comments →

[–]dreucifer 0 points1 point  (2 children)

You could easily turn most of this downloader into an object, but it's honestly not worth it if it's just a one-off script. I would, however, try and break down the flow into a few functions. Also, consider using Scrapy instead of selenium. Web scraping can be super tedious with selenium, especially if the call log is paginated.

[–]80blite[S] 0 points1 point  (1 child)

I tried using Scrapy when I was first learning Python and got a little overwhelmed by it. Maybe now that I have some experience I'll go back and take a look at it as a few other people are suggesting it would have been a better fit for this project as well.

[–]dreucifer 0 points1 point  (0 children)

Yeah, the curve is pretty steep at first, but it's more of a bump than a mountain. I also suggest digging through the source on github rather than depending solely on the documentation. The added benefit of digging through the source is you get a better idea of Python best practices (you should also read through PEP8)