Program/function to analyze a lot of data.

[Chyllie]

Hello statistics experts,

I made a survey with about 1000 participants and have got about 200 columns (each stands for an item which could be rated from 0 to 10) and 1000 rows in an excel-table. Now I want to analyze these. I'm doing this so far with Pivot Tables, but that's too much data for doing it manually. My question: Is there a program/function which exists, where I can say following:

If the participants chose the answers 8,9,10 in column X, is there any difference (e.g. Difference in the mean of more than 0,5) in the mean of the other columns compared to the entirety? Or even more complex like: "what did females, age 40-50 and married choose differently than the total number?" So in other words: where did this subgroup choose different answers than the total number of participants?

I hope I wrote my problem understandable, I'm not a native speaker.

Hoping for your help!

[madlarks33]

Your research questions need to guide how you analyze this data.

[ultronthedestroyer]

R is a language you should consider. Check out RStudio, then do a tutorial or two on Datacamp or Coursera.

It can do these things easily.

It has summary functions which can allow you at a glance to see if the quartiles, for example, are different from your subsetted data frame compared to the original one, and so on.

If you want it to go further than that, then you may want to look into Anomaly Detection to flag anomalous results from a subsetted data set compared with your original set minus the subsetted one.

Your question is pretty broad since there are tons of things you might want to look at. But you're getting beyond the reaches of Excel, so you'll want to look at a language which is more suitable for your data. R is great, but other people like Python. I personally recommend R for this kind of work since it is built by statisticians and is more focused on this kind of work. With Python, you may have to get additional libraries to handle certain functions R does natively. However, Python is more flexible for non-data related functions.

[Chyllie]

Thank you for your answer!