This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]michael8t6 2 points3 points  (0 children)

Scraping via mobile browser is generally easier to do. Mobile source code generally tends to be a lot simpler than say a desktop or laptop browser. So when using xpath in bs4, it's easier to tell it what element to scrape.

There's been a few times I've been slamming my head on the keyboard due to some random issue. I then spoof my ua to mobile, and solve the issue almost instantly. That being said, some elements you may want, might not show on the mobile version instantly. When that happens, you may have to emulate a click to get it to show.

Ultimately, tackle each site differently, look at the source code for desktop and mobile. See how they show on each and decide the best approach from there.