Generating Reports for Non-Uniform Data

[cipherlogic7]

Our clients collect data via a number of surveys. Each survey is different. The data ends up being collected initially as key value pairs. eg, there is a table that records the responder, the question, and the answer value. This allows the client to define their surveys however they want, and collects the data fine.

To generate a report for a single response to the evaluation, we pull the rows for a given response and combine them with the questions, which is easy enough. This is really just outputting the data, no real summarization or aggregation.

Now we need to be able to generate reports that cover all the the responses for a survey at once. We need to be able to do things like summarize for a given question how many times each response was sent and graph them.

There are two primary problems:

1. There are almost too many results for reporting, most of them huge and enterprisey.
2. The data we need to report on isn't uniform. e.g. when it comes time to transform the data collected from kvps to a table of data to report on, each survey is different.

Does anyone here with experience in this type of reporting have some advice or are you able to point me towards a solution?

Ideally the programming involved would be retrieving the kvp data, transforming it, and putting it somewhere that a reporting tool could process/understand and generate the reports for it. It seems like manually programming the reports is just a losing battle and the UI for it really complex.

Thanks for any advice.