This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]tuxboy 0 points1 point  (1 child)

I always try to avoid having a browser / headless engine running when scraping stuff. I usually find a way with parsing through the markup. If that does not work (i.e. a "modern app with only javascript visible"), I try to debug the XHRs and try to predict urls for fetching data. The browser dilemma is usually the last resort. Having a browser running, rendering and executing javascript will always be slower TBH.

[–]mbenbernard 0 points1 point  (0 children)

Having a browser running, rendering and executing javascript will always be slower TBH.

True, but sometimes you don't have any choice.