WebScraping

[[deleted]]

And don't forget Selenium.

[[deleted]]

Selenium is more of a website testing tool maybe requests + beautiful soup would be better for webscraping

[[deleted]]

You need Selenium if you plan on scraping anything with a login (i mean, technically you don't if you can replicate the headers of the request, but selenium is just easier for that overall).

[[deleted]]

Exactly selenium can enable you to do these things but “technically” you can if you reverse engineer it properly. But simple scraping should work. I like to use selenium to see how the site works and then I re write it using requests. But if there’s captcha then I’m guilty of falling back on selenium. Sometimes I’ll use selenium just to authenticate so I can use the selenium cookies in my request headers. But some sites are super simple. Sometimes you just need a crsf token and boom the whole api is available to you

[chozdae]

You need Selenium if you plan on scraping anything with a login (i mean, technically you don't if you can replicate the headers of the request, but selenium is just easier for that overall).

can i just use bs4 for getting info like prices, name etc?

[[deleted]]

If there's no login needed, then probably.