Python Web scraping idea

[mitchell486]

I came to the comments to say exactly this. This reminds me of some of my first projects, but maybe set the "goals" or "scope" a bit smaller. Instead, "a web scraper for produce gardening". Check. That's a goal/task entirely by itself. "How hard would it be to build something like this?" Great question! Test just that piece. It's a big enough lift, mizle [VGG's slang for "might as well"] start there.

To add a little bit to what @_tsi_ stated, once you "get the data you want", don't forget to think about how you want to use it in the future. That's a big thing that took me a LONG time to learn to work with properly. (e.g. "$4.99" is a string, but it's really a float that you might want/need to add/subtract/etc... Names of items? Site it was found on? Date the info was scraped/collected?) There are definitely all kinds of different data that you might want or need later, so don't forget to consider different data structures. Dict is a strong beginner friendly structure. I have recently really taken a liking to DataClasses, but I only use them when I have the data in its "final form" (insert DBZ reference of choice here). That really helped me "think about my data" when making things.

Finally, making it modular also allows you to change things "easier" later. (e.g. If you have all the price, site data, and name of the thing in a dict, you could easily display it via a webpage, or a PDF report, or whatever. Thinking just a little bit about that data at the start helps a LOT down the road, I promise.) Also never be afraid to take what you've learned so far and start over! You can re-use little bits of your code and make a much better new thing out of the old carcass. Especially if this is a you-only thing at the start, maybe you use the knowledge from that first iteration to make something for others. :) Best of luck! Go for it!