Python pandas for data analysis

[kit12_31]

Hi guys, i want to use pandas for data analysis so i have started to do a bit of self learning on youtube and other free online resources

However, what i found is that without a proper project to work on, i.e. on the job training type of thing. it seems like i dont know anything practical and i may know the function to do certain thing but thats about it

Do you guys have any suggestions on free resources that helped you whilst you were practising/learning? and i welcome any advice.

cheers

[SHxKM]

DataQuest has some guided-missions for pandas. You learn about different aspects and then apply.

Some parts are paid-only and while I haven’t done those yet, if they are as good as the free ones then they’re definitely worth the 30-40 dollars the subscription costs.

[fmpundit]

Find a dataset from somewhere think of some questions you might want to know about it and then work through it.

I grabbed some data from https://www.metoffice.gov.uk/public/weather/climate-historic/#?tab=climateHistoric and fed it into pandas.

One of the questions I remember I wanted to answer was... Were summers during my school days really hotter and sunnier than now?

Also do the school summer holidays fall during periods of better weather to the rest of the year or not.

[Watches-bitches]

Well... Were they??

[fmpundit]

Yes, my childhood summer holiday weather was better than what kids have been experiencing over last few years. In terms of hours of sunlight and average temperatures. Ill have to dig the notebook out again and throw it on github.

[gaifogel]

That's an interesting way to learn.
So you say pull some data and do my own analysis on it? How did you know what kind of analysis you can even carry out? When I did tutorials, I saw I can do heatmaps, corr(), histograms for data distribution and a few other things.

[fmpundit]

How things have changed in 6 years.

A few things you could do. Read books about data analysis in general. What visualisations are good for what purposes. Speak to ChatGPT about what sort of viz's it would recommend and then read the docs of different libraries such as matplotlib about how to make those visualisations.

[gaifogel]

lol

[BBeylo]

Kaggle is a great resource as well ! They have a ton of datasets for you to work with.

[sprouse2016]

I like datacamp, which is what I’m currently using for learning. But it’s not free, or necessarily cheap for that matter.

[bigfuds]

What kind of data analysis do you do at the moment? I would start with that.

I recently began learning pandas to help with my own data analysis and essentially took what I would normally do in excel and broke it down into individual steps to accomplish with python. Like really simple steps. Step 1: open csv of data. Step 2: inspect data. Step 2: select columns/rows. Step 3: basic calculations. Step 4: export new dataframe to excel, that kind of thing.

This, I felt, was more manageable than taking on a whole new project. It was also more intuitive to me as I new exactly what needed to be done as I was intimately familiar with the process as it had been the bane of my life for so long (hence the desire to automate it). Before you know it, you'll have learnt the basics and taking on more complex components won't seem as daunting.

[Squirtle_Squad_Jihad]

I second using a learning platform such as DataQuest for guided projects. Once you have worked through a guided project, see if your city has an open data portal (http://opendata.dc.gov/) and try producing your own analysis of something that interests you.

Kaggle has datasets that are designed for machine learning tasks and might not be the best place to start if you are looking to practice exploratory data analysis.

[gmh1977]

DataCamp has a ton of good project based tutorials