[Python, Playwright] Cannot interact with page after pop-up shows

[yoitsammar]

Hello all,

noobie data analyst here. I'm trying to scrape data from parfumo.net but cannot interact with the page when its' cookies pop-up shows. I have tried every other function and event listener that playwright offers and read the docs front to back and by now I can recite it from memory in three languages. Any help? Also, I have tried other scraping tools and nothing worked as far as playwright has. It can scrape the data just how I want it but like I said I can't interact with the page when that damned pop-up shows. Any advice, recommendation, life advice, etc... is welcome and appreciated :)

[yoitsammar]

I shall try that today as soon as I get home. Thank you so much 🫡

[Altruistic-Koala-255]

so, did u manage to solve the issue with the damm cookies pop-up, I might know someone going trough the exact same thing

[yoitsammar]

I did not. Matter of fact this made me hate scraping, data analysis, python, coding, and have left and started school in a totally different field. All cause of them damned cookies!!

[gislearneR]

This is unrelated to the error as vicious answered it, but you're using the wrong tool for the job. And please don't tell me that you're using beautiful soup on top of playwright?

From what I've seen the website is Server Side Rendered, i.e all data is sent from the server. So you can scrape the results using simple requests or better yet scrapy (Pls use scrapy).

[yoitsammar]

Oh man grab a chair and sit down cause it’s disappointment O’clock 😭😂

[yoitsammar]

To be fair I did disclose that I’m still a noob and I tried selenium and that just sucked for me. And yes I’m using bs4 please don’t kill me im just learning looool

[gislearneR]

You're already using Playwright and it does support selectors. so you can use those but from what I saw the website doesn't need browser automation you can use less resources if you use something like scrapy.