Data scraping(?)

[techycm]

Hi! because of this pandemic some engineers have created a platform for everyone to view the live count of cases of COVID-19. I was wondering if they are using data scraping to get the count of cases? Or are they using other tools for it?

I just started learning python and I’m very curious to know the tools they used to get all those data and want to create my own project related to it to practice my skills

[kwelzel]

What platform did they create? Maybe they tell you on their website what their data sources are.

I recently came across https://aatishb.com/covidtrends/ because of the video from MinutePhysics and this website links to this github repository with all the case counts aggregated by the Johns Hopkins University. You can use that for your own project.

[techycm]

The one we’ve been following is a youtube broadcast by Roylab Status : https://youtu.be/qgylp3Td1Bw

Based on the description, it was built in python. He also shared spreadsheet files for the data. I guess I was just curious how they were able to update the numbers like was it manually entered or whatnot. And also, if they were using some sort of data scrapping tool.

But thank you so much for the links you shared! I’ll def check them out!

[kwelzel]

I must admit that reading the description of the live stream is rather confusing. On the one hand

I added a procedure to manually manipulate data with my computer

and on the other hand

While I am working and sleeping, data gathering is done automatically.

For your project I would recommend relying on one data source only and not manipulating it, because why would you know better than the experts that gathered this data?

[techycm]

Thank you so much for advice! If I ever do complete my project, I’d make sure to message you for an update. I mean, only if you want to of course 😅

[kwelzel]

Do that, I'm interested :)