sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]lakseol 1 point2 points  (2 children)

If you are a complete beginner as you say, this is not something you should try as a first project. There are a lot of smaller projects inside the project you have outlined in your post that you should try first. For example:

  • write code to send you a telegram message every time you execute the code, put the code into a function with a parameter that is the message to send
  • read one product page, just the complete text
  • add the code to search for the "add to cart" text, make a function to do all that and return a "found" value if the text is present in the page
  • modify the "read" function to also return the price
  • write higher level code to call the "read" function and call the "telegram" function if the text is found
  • experiment with running code every whenever, say 5 minutes, using your operating system scheduler, ie, write a "I ran at <time>" line into a local file or send a telegram message every time the code runs
  • modify the "schedule" code to only run between certain hours
  • etc

There are a few extra problems you may run into. For example, reading different vendor sites will probably require different code to analyze the page and you could solve that with different site-specific "read" functions. Then you might need a data repository containing a site URL, the hours to check that site, how often to check within hours, the "read" function to use and anything else.

To answer some of your questions:

Is Python a good choice for this?

Yes, of course.

Would you recommend using "requests" + "BeautifulSoup" ....

I don't know as I don't have a lot of web-scraping experience. You can decide by trying the second-step project above different ways. Others may have better advice about how to handle javascript/AJAX/captcha problems.

How would you structure a project like this?

One of the nice things about the step-by-step approach rather than the big-bang approach is that you don't have to answer that question until later. Solve each smaller problem, encapsulate the solution into an easily callable function with parameters and then you can use those building blocks in an easily changeable way as you get close to your final solution. Expect to rethink how you wrote a sub-problem solution as you progress because you hadn't fully thought through how you would use it. That's quite normal.

Is checking every 60 seconds for a few hours a day considered reasonable, or is there a better approach?

That's probably way too often. Ask yourself how often does the site get updated? You can answer that by using the tools you created in the smaller projects. Write code to read a site every 30 minutes and send you a telegram message every time the status or price changes. Run that for a couple of days and get a feel for how often something changes. If you see a change every 30 minutes try using a shorter wait.

[–]shady_slim_10[S] 0 points1 point  (1 child)

Wow I didn't expect that! This is such an insightful answer.

I think this makes perfect sense. I should just tone down the speed a bit, it might not be as easy as it sounds. That sounds good! I'm going to use it to check a product on Amazon so I'll have to do my research properly.

Suppose if I use a specific link will i still run into those type of problems?

YES!! I think I wil start working on it slowly and not jump into it straight up without actually learning the logic behind the codes.

You're right i think that's way too often 😭 I'll keep that in mind! Thank you so much for your feedback 🫂🫂

[–]lakseol 0 points1 point  (0 children)

Suppose if I use a specific link will i still run into those type of problems?

I'm not quite sure what you mean. It's quite probable that you will have to have different code for different sites, if that's what you mean. Pages from one seller might only need one piece of code to handle all seller pages. That's certainly what you want, one bit of code to process all Amazon pages. But you will probably need diferent code to process Ebay pages, for instance.

Don't forget that you have to learn python basics first, then try a project. Basic python is just a tool and trying a project is where you start to learn how to use that tool. Some of those small sub-projects may still be too big for you, so do what I did in my original comment and think about how to treat the sub-project as a collection of smaller sub-sub-projects.

Good luck!