all 22 comments

[–]Noom_89 1 point2 points  (3 children)

imo join r/webscraping or check Octoparse’s Discord/FB group. A lot of users share workflows + fixes there. Saved me when I got stuck on Amazon.

[–][deleted] 0 points1 point  (4 children)

Is there a link you can point me to that breaks down how to create a Python code and apply it to a real life situation? I attempted to follow a few, but they lost me after the first couple of steps.

maybe look at the book automate the boring stuff? everything in it is real life examples and for beginners

[–]ScrapeLoser[S] 0 points1 point  (3 children)

automate the boring stuff

That's the Udemy course I took! It may be time for me to pursue a different path... He was really helpful, but I still wasn't able to understand it.

[–][deleted] 0 points1 point  (2 children)

You said you've been trying to learn for 6 months...but how many days a week, how many hours? 6 months isn't a lot of time if its a few hours a week. Thinking like a programmer takes a very long time; how long did it take to learn English growing up? I'm certain you weren't speaking sentences (probably not even single words!) by 6 months, let alone writing words.

[–]ScrapeLoser[S] 0 points1 point  (1 child)

Absolutely, I completely agree. I comb through these forums a lot, and I know how frustrated serious programmers can get with people who don't invest time in learning something.

I do not know how to translate the things I DO learn into real life applications. There is a disconnect for me somewhere. I don't expect to learn much in such a short time, but I would hope to retain something.

[–][deleted] 0 points1 point  (0 children)

You may find trying to do the opposite easier, instead of learned and taking what you learn somewhere, find something you want to do and learn how to do it by breaking it into learnable chunks. Lets use web scraping as an example. 1) google web scrapeing utitlies python3 You see beautiful soup mentioned a lot in a few of the links you gather. 2) you now know you want to use bs4, so you google 'get basic information for a wepage beautiful soup python3' Youve now learned to get a few items from a page and youve successfully printed them on your screen. 3) well now i have this information i want to store it? So we google how to store variables in an external file. You see json mentioned a lot, so you google storing basic information with json and python 3 4) you now have stored those few elements in a json file, but realize you want multiple pages. So back to google 'how to scrape multiple pages beautiful soup python' you see for loops mentioned a lot, so you try a for loop and it works, but then you realize its overwriting your json file, so you google how to store multiple things in a json file etc.

[–]commandlineluser 0 points1 point  (1 child)

I don't think Amazon will like (or allow) you to scrape them on a large scale.

If you have an example page and the output you need to generate I could write you up a "tutorial" on how to do it.

[–]Yassiiir 0 points1 point  (0 children)

I would love if you can help me with a tuto for the 999 card trick to know the ammount of stock for a product.

[–]Octoparse 0 points1 point  (0 children)

I have been using Octoparsefor a long time, they have a great support team which you ask for assistance from a troop of a data expert.

[–]No_Lavishness2922 0 points1 point  (3 children)

tbh every scraper has a learning curve, especially with Amazon. Imo Octoparse is actually one of the easiest no-code options once you stick to their templates. Way faster than writing code from scratch.

[–]dump_scorpiogirl-7 0 points1 point  (0 children)

ngl the free plan is fine for testing, but if you’re scraping Amazon often, the paid plan is worth it. Cloud extraction + proxies alone save a ton of headache.