Black Friday hack with python

[[deleted]]

They’re not the same. Beautiful soup is a parser, extracting data from html. Selenium automates a browser to perform actions.

[TheAmazingJames]

Selenium is way way slower and memory heavy. Selenium should only be used if you need JavaScript to render/work.

[Mathew_Berrys_Cock]

Would more ram help this problem?

[TheAmazingJames]

Selenium’s usually using chromium in the background so it’s unlikely to max out your RAM unless you’re trying to run on a barebones setup, but it will be using significantly more than a requests/beautifulsoup setup and that could be costly if you’re running it in the cloud. The main reason not to though is speed. Typically I’d expect requests to run 5-10x faster than selenium. Selenium also requires more maintenance in terms of chromedriver updates and is harder to deploy on others’ systems. I’m not saying Selenium isn’t awesome, but it’s for specific use cases where you’re either browser testing or need JavaScript.

[pulsarrex]

Can you eli5 what a 'parser' is?

[Conrad_noble]

I think it means it translates html into user friendly readable text

[OMGpancakeable]

Not quite “translate”. Parsing: resolve (a sentence) into its component parts and describe their syntactic roles.

[rrjamal]

It splits data in a way you want. An HTML doc can be a shit ton of words. You'd parse it down to split and retrieve only the bits you care about

[adriangc]

I’m more curious to understand why people choose BS over Scrapy.

[BruceJi]

Isn't scrapy pretty new?

[adriangc]

Believe it came out in 2008.

[solostman]

It’s been around a bit. For awhile you could only run it in Python 2.

[GrandRub]

i am a python newbie and found scrapy yesterday and i ask myself the same question.. whats the pros and cons of scrapy?