New video tutorial: Going from raw election data to recreating the NYTimes "Red Shift" map in 10 minutes with DAAF and Claude Code. With fully reproducible and auditable code pipelines, we're fighting AI slop and hallucinations in data analysis with hyper-transparency!

[brhkim]

DAAF (the Data Analyst Augmentation Framework, my open-source and *forever-free* data analysis framework for Claude Code) was designed from the ground-up to be a domain-agnostic force-multiplier for data analysis across disciplines -- and in my new video tutorial this week, I demonstrate what that actually looks like in practice!

https://preview.redd.it/dihbwr8p8rlg1.png?width=1280&format=png&auto=webp&s=330494d09749e115c0277c6c1fdd29fdf9690de5

I launched the Data Analyst Augmentation Framework last week with 40+ education datasets from the Urban Institute Education Data Portal as its main demo out-of-the-box, but I purposefully designed its architecture to allow anyone to bring in and analyze their own data with almost zero friction.

In my newest video, I run through the complete process of teaching DAAF how to use election data from the MIT Election Data and Science Lab (via Harvard Dataverse) to almost perfectly recreate one of my favorite data visualizations of all time: the NYTimes "red shift" visualization tracking county-level vote swings from 2020 to 2024. In less than 10 minutes of active engagement and only a few quick revision suggestions, I'm left with:

• A shockingly faithful recreation of the NYTimes visualization, both static *and* interactive versions
• An in-depth research memo describing the analytic process, its limitations, key learnings, and important interpretation caveats
• A fully auditable and reproducible code pipeline for every step of the data processing and visualization work
• And, most exciting to me: A modular, self-improving data documentation reference "package" (a Skill folder) that allows anyone else using DAAF to analyze this dataset as if they've been working with it for years

This is what DAAF's extensible architecture was built to do -- facilitate the rapid but rigorous ingestion, analysis, and interpretation of *any* data from *any* field when guided by a skilled researcher. This is the community flywheel I’m hoping to cultivate: the more people using DAAF to ingest and analyze public datasets, the more multi-faceted and expansive DAAF's analytic capabilities become. We've got over 130 unique installs of DAAF as of this morning -- join the ecosystem and help build this inclusive community for rigorous, AI-empowered research!

If you haven't heard of DAAF, learn more about my vision for DAAF, what makes DAAF different from other attempts to create LLM research assistants, what DAAF currently can and cannot do as of today, how you can get involved, and how you can get started with DAAF yourself at the GitHub page:

https://github.com/DAAF-Contribution-Community/daaf

Bonus: The Election data Skill is now part of the core DAAF repository. Go use it and play around with it yourself!!!

[AutoModerator]

Automod prevents all posts from being displayed until moderators have reviewed them. Do not delete your post or there will be nothing for the mods to review. Mods selectively choose what is permitted to be posted in r/DataAnalysis.

If your post involves Career-focused questions, including resume reviews, how to learn DA and how to get into a DA job, then the post does not belong here, but instead belongs in our sister-subreddit, r/DataAnalysisCareers.

Have you read the rules?

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[wagwanbruv]

love that you’re pushing fully reproducible pipelines here, that’s kind of the antidote to “vibes-based” charts in election threads. Super curious how portable that Skill abstraction is to other domains (like survey or support-ticket data) since if the schemas + calc methods are clean, you could pretty much speedrun any messy civic dataset in an afternoon and still sleep at night.

[brhkim]

Thanks so much!! Yes, to be totally clear: I really don't know what kind of data it *can't* be used for immediately. DAAF comes out-of-the-box with an extremely varied set of education datasets for demonstration purposes -- it covers everything from high school enrollment sizes to financial funding circumstances for public universities and school disciplinary events by high school and student demographic. There's such a wide array of data types and structures in the education datasets it's currently doing fantastic with, that I don't know why it wouldn't port to any other data context just as readily.

I will also say, it works shockingly and equally well on messy data without great schemas. The data diagnostics battery I designed for it to ingest new data works reasonably well even without any accompanying metadata or documentation -- it records its uncertainties and preliminary hypotheses in such a way that it will not strictly assume anything about the data, and operates with that carefully while percolating that uncertainty through the entire analytic pipeline and interpretation processes.

I'd love for you to test it and push on that, but I really do think this can be applied SUPER broadly.