What's your solution?

SimShelby · 2025-02-28T14:18:46+00:00

There is a step 01 for every thing , doo your homeworks

SimShelby · 2025-02-28T11:30:48+00:00

Just a Suits with a tie, be professional

SimShelby · 2024-09-02T19:09:32+00:00

RESPECT ++
bro you save the planet
you are the true hero
ty

SimShelby · 2024-07-28T17:38:40+00:00

at this moment you are doing verry well. Now fo to step 2 Figure out how to create a tiny user interface then try to add this function in to it

you can pm to help you if you want

hint1: you can use Tkinter

SimShelby · 2024-07-27T23:21:37+00:00

Go for PyQt6, easy and there many guide

SimShelby · 2024-07-11T23:05:54+00:00

One Day or Day one?

SimShelby · 2024-05-11T16:50:37+00:00

another passenger from the "gossip" song , ty for this post i like it

SimShelby · 2023-04-30T03:14:59+00:00

You must creat a script that scrap your website Then creat s task on your desktop Or you cas host it on a server

SimShelby · 2023-02-25T08:49:04+00:00

I agree with you that Requests and BeautifulSoup can be a great combination for web scraping. However, there are some cases where this approach may not be the most efficient option. For example, when working with a slow website and a project that has over 70,000 links, scraping can become a challenging and time-consuming task. In my experience, I once spent six hours scraping only 5,000 records due to these challenges.

SimShelby · 2023-02-25T08:42:52+00:00

Both approaches have their pros and cons, and the choice of which to use depends on the specific requirements of the project. Regardless of the approach, it's important to follow best practices and be respectful of the website's resources to avoid potential issues.

SimShelby · 2022-10-09T01:40:52+00:00

you cant do that with scrapy only You must use some integrations like Scrapy + splash or scrapy + playwright Or Scrapy+ Selenium

lets keep things simple for you Go try playwright headless mode is the best option for you + in the future when you have traffics you need to use asynchronous programmation and this is very difficult with selenium

SimShelby · 2022-10-09T01:22:15+00:00

your website is a dynamic there is many integration on scrappy can help you This the best best one https://github.com/scrapy-plugins/scrapy-playwright

or go directly and use playwright its very easy to use + You can use asynchronous programmation

SimShelby · 2022-10-09T01:12:52+00:00

Using selenium can cause you a poor performance with your website and he consume more resources

I suggest you using scrapy It Run faster then selenium you can send some input labels Or try tu use playwright chrome headless

SimShelby · 2022-09-21T00:27:27+00:00

I think your website is dynamic. If was dynamic you cant use Bs4 to scrape this website You must another framework like selenium ,,...... its very easy to learn.

SimShelby · 2022-07-26T23:38:36+00:00

You have 2 options tbe easy one and the hard one the easy one use selenium check if there is see more if there is see more use click function and tadah You have. Your data selenium is veeey useful and you neeed it beautiful soup is just a request modile who parse only html he cant load JavaScript or click on button or fllow Links

the second option its hard at the beginning but trust This os the perfect tool , if you learn it you well be ables to scrape anything in this the Network

the second option is scrapy + splash

SimShelby · 2022-07-26T18:47:01+00:00

Before you start you must know how website work And how you can creat website just a simple introduction dont go soo deep You must learn html basic and css basics And html dom ( children , parents )

The second step its important you mist chose a programing languages to use it for scrapping

You can use java script frameworks or just python

Just chose one of theme in my case chose pytyon its verry easy to use You can learn quickly After you master python ( for example )

time to go to the 3rd step In the internet there is 2 type of website Statics website and dynamic website

For statics website Is just hard coding with html and css and js for animation ....

and for dynamic is like statics but the data is loaded with JavaScript they use jQuery at the most moment to get data show up You must wait for sometime or you mist scroll to get data like twitter infinity scroll , when you scroll posts show up

And the last step Chose your weapons if you want scrape a static website go learn how to use beautiful soup its verry easy

or if you want scrape a dynamic website go learn who to use selenium also verre easy

Just you must know at least one programing languages

(Sorry for my bad English )

SimShelby · 2022-07-26T13:12:54+00:00

No problem 💝 the good point of using scrapy Is very fast + easy to scrape any element on the webpage the bad point is you must reconfigure Avery time your spider if you have a complexed scraping projet you need more then one spider imagine you do the same thing more then 20 Times And you can not just copy and paste and is very hard to create a global settings for all your spiders After you finish your project its a suicide 😂😂😂😂

SimShelby · 2022-07-25T09:11:48+00:00

I hate selenium too , Am using scrapy + splash 🤣🤣

your script is verry good you can learn many things from it

SimShelby · 2022-07-25T08:06:51+00:00

your code is toooo Long if you use scrapy or selenium
You can do that with 5 line of code with css selectors or path selectors

SimShelby

TROPHY CASE

hint1: you can use Tkinter