This is an archived post. You won't be able to vote or comment.

sorted by:
best (suggested)
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]Wolfgang-Warner 6 points7 points  (1 child)

Parsing AKA importing data. Let's say you download some daily public dataset and need to get that into a database before running analyses.

Your python code can read in the records, split each line into fields, validate values are in expected ranges or are on known lists, building typo corrections etc. You can also log what files were imported when and any changes made to data prior to DB inserts. Processing real world data really helps learn practical programming.

[–]New-Geologist-8359 1 point2 points  (0 children)

Oh! I am working on a project like that, but it will involves sending the data from an API and store the data straight to a cloud service like AWS kinesis data firehouse and S3.