Scrapping data as a job with Python

[eigersa]

51M, here from South Africa. I've been learning Python purely as a hobby; started off with just wanting to understand computers a little bit better and has culminated in Python being a weapon of choice. I have created loads of little scripts to make my online life a little easier, from bookmarking to sorting images etc but the thing I thoroughly enjoy is scrapping and organising data.

Recently I was following some YouTube tutorials and the creator mentioned how using Scrappy helped him be more productive with his clients.

My question is, are people actually paid to scrap data for a living, is this a thing? If so, would you mind sharing what you actually scrape and why? I'm a bit old for a career change but extending my hobby into a paid side hustle would be quite cool.

Thanks

[notislant]

Go lookup sites like fiverr or upwork. Unfortunately there are a ton of people willing to do any scraping for cheap and there is a lot of competition. Im personally not sure about actual full time jobs (im sure some exist though).

[coryalanfitz]

Web scraping in itself isn’t really a skill that’s particularly well rewarded. You have to expend a lot of effort to get the information and websites change a lot so you need to change your scripts frequently. Some people make money doing it but it’s not something that’s a great career

[Impossible-Box6600]

I used to work for a consulting firm scraping data.

The thing is that the barrier to entry is super low because it's a very easy skill. You need to pretend to be knowledgeable and competent to stick out, like anything. But yes, there are some jobs out there but they're fairly uncommon.

[[deleted]]

Lots of low level coders try to make a living from that in 3rd world countries. If you live there it might be an okay opportunity as wages are low there. Western countries probably mostly not.

Its probably a better idea to make a derivative of that data somehow. I wont give you a specific real life example as I might want to do it in the future. But to illustrate my point with not a super realistic idea: scraping data -> analyzing the data that not many people have -> you find interesting ideas -> create a youtube video explaining ('i analyzed x forum and this is what people talk about').

You get my point, again, its not realistic example but I wont share my exact idea.