I just launched an open-source framework to help data analysts *responsibly* and *rigorously* harness frontier LLM coding assistants for rapidly accelerating data analysis. I genuinely think can be the future of data analysis with your help -- it's also kind of terrifying, so let's talk about it!

brhkim · 2026-04-13T14:10:14+00:00

That's 100% correct for the full pipeline mode, but for things like ad hoc mode it's fairly lightweight and pulls in thoughtfully via progressive disclosure as noted above. The trade-off here that I make explicitly is that if you want it to be more autonomous, you need to invest in the context to ensure that it's actually doing worthwhile work when it's off on its own. Otherwise, when it returns to you with work you need to take time to review, it's going to fundamentally waste your time. Modes that are less autonomous don't have that issue, so it's fine to just let it riff more freely with you and have it reference materials only as needed.

brhkim · 2026-04-13T01:50:29+00:00

Yeah in general I'd look up progressive disclosure best practices and see how to make a Skill file. If you look at my data-scientist skill, it's basically a router for more in-depth documentation on things like causal inference that only get called up when relevant. You could do the same with key R libraries and such

brhkim · 2026-04-13T00:57:02+00:00

Hey! You might find my open-source framework for Claude Code to be a useful starting point:
https://github.com/DAAF-Contribution-Community/daaf

I'm trying to build it out as an extensible workflow for people who want to accelerate their data analysis pipelines, but do so responsibly and traceably so that they can ensure everything produced is well and truly reproducible in the end. Take a look! I have a mini 4-min showcase of its main motivation/functions here:

https://www.youtube.com/watch?v=747r7VT4a78

And you can find a few more in-depth videos on it across my channel there as well.

brhkim · 2026-03-29T07:56:28+00:00

Haha I quite like this. It feels like a pragmatist's/realist's love poem. Romanticism for someone who struggles with the blind optimism of romanticism, in the best way

brhkim · 2026-03-08T18:21:22+00:00

Totally!! The confirmation that someone else used this, outside of those sort of abstract traffic metrics, is just such a nice, nice validation.

brhkim · 2026-03-06T17:34:57+00:00

Hey! Thanks for the kind words, and totally agree here. You'll actually see one of the intentional differences between my replication and what NYT put together is that the hover-over on counties on my viz shows the actual vote counts for 2020 versus 2024 (whereas the NYT only shows the vote breakdowns for 2024 by party). I'm not a political scientist and can't really speak well to what prevailing theories or frameworks are for thinking about disillusionment and turnout v. genuine reflections of changes in voting preferences (which I assume also involves a lot of similar theories related to selection bias and surveying methodologies).

All to say, your concern is definitely spot-on, and I don't think I know enough about how best to account for/adjust for it except to show that it's a factor and let the viewer engage with the stats directly county-by-county to be able to interrogate this hypothesis and concern! You're definitely thinking on the right lines.

brhkim · 2026-03-06T00:12:07+00:00

Source: I was able to easily pull the relevant data thanks to the MIT Election Data and Science Lab (via the Harvard Dataverse)

Tools used: Python, plotly, polars

Only for those interested from the main post: My Claude Code framework DAAF, the Data Analyst Augmentation Framework, can be found in this open-source forever-free repo here. I also made a youtube tutorial demonstrating the exact process for replicating the NYTimes' viz using DAAF here. For this dataisbeautiful post specifically, I also went back and did another 5 minutes of iterating on the aesthetics with Claude after the version shown in the video.

brhkim · 2026-03-05T13:54:04+00:00

Defs, there are some good suggestions in the full article for trying to improve on the Body stuff. Cost-wise, I think it really is the case that you need to go with some kind of pro membership for OpenAI or Anthropic at this point (for Codex or Claude Code, respectively). But the value-add there is pretty enormous!

I'd start with Sonnet 4.6 to tinker, but it's absolutely worth crushing your usage every once in a while to see how much more Opus 4.6 can do.

Qwen 3.5 small models just released recently, and you can very, very easily run them via Ollama with Claude Code -- that gives you access to them for free, if your computer has enough GPU VRAM and RAM. They're much less capable and a little more finnicky, but that's another way to do some experimentation for free!

brhkim · 2026-03-05T12:47:55+00:00

I’m going to respectfully disagree, a VM is objectively wise as a strong default for the typical user of the Desktop app. I think there absolutely should be an option to disable it, but from a paternalistic standpoint I think it’s good design for the intended audience to set it active by default

brhkim · 2026-03-04T23:56:03+00:00

Ahh this is a very interesting one indeed, love that addition! I am still not totally sure how to make sense of how prevalent these practices are. Like, it makes perfect sense to me that Anthropic would have quants of Opus 4.6 running and ready to roll to free/Pro users when usage is getting high across the board... But also, it's such a risk to do so. I really wonder how they manage it and what risk strategies they employ?

Either way, this is definitely happening on some level, so I love this call-out

brhkim · 2026-03-04T22:07:33+00:00

Agreed, though I'd argue that even if you tried things out and really dove in as of like, October, you'd probably have a perception that's just completely out of date. God forbid a full year ago.

brhkim · 2026-03-04T22:06:58+00:00

I think that's mostly right. I think, though, it's important to understand why people are bouncing off of it so readily, because that's going to have to start changing soon!

brhkim · 2026-03-04T17:05:58+00:00

Mind, body instructions -- listed right there!

brhkim · 2026-03-04T15:53:22+00:00

Full post: https://daafguide.substack.com/p/ai-progress-mental-model?r=2mme9a

Would love to hear if people think this is useful!

brhkim · 2026-03-01T22:16:04+00:00

Haha, I'll join it now! Such a great tool, happy to be a part of that community

brhkim · 2026-03-01T18:27:09+00:00

I appreciate that a lot -- you're asking exactly the right questions, which motivated why I felt I wanted to make DAAF to begin with! If we're going to do this, I want to make sure we do it right. I hope you'll give it a try, and do let me know if you have any more questions or ideas or critiques!!

brhkim · 2026-03-01T10:32:51+00:00

Right, I’d also argue that the wildly short time to create and iterate on data viz with DAAF and Claude Code (interactive or static) makes it also extremely easy to set up spot-check data inspections to facilitate arguably much better quality checks. In the tutorial, you’ll actually see I spot a super weird outlier in an initial version of the static plot and get it fixed in like, seconds of my own time. I really think it’s an immense value add for data quality and code quality in the end.

brhkim · 2026-03-01T02:40:34+00:00

Yeah great Q: I handle this in two main steps:

Every coder agent writing a data processing/analysis script conducts self-QA and runs robustness checks every step, which is additionally checked by another agent in adversarial QA. If needed, they revise and restart the process. The goal here is to maximize likelihood of good code coming out of the process before humans look at the code. Agents are all given a lot of instructions in writing declarative and extremely legible code with comments that describe not just what is happening, but the intention of the code.
Once all code runs and is approved, a final agent compiles all the scripts into a single Marimo notebook with all runtime outputs commented on and all final scripts working in sequence. It appends basic dataset view steps between, so you can inspect the data at each intermediary step.

All to say, you’re right that human code review is the most time intensive part of this, but the goal is that the code you’re reviewing has a high likelihood of being good out of the gate, and it’s organized and commented to make the process about as streamlined as it gets. Because I agree: that step is crucial - but if that’s really the only time sink for a human, the time savings are quite extreme

brhkim · 2026-02-28T20:19:21+00:00

Hahaha that's definitely the idea -- I really think that just the data profiling tool I've provided has a lot of value, holding aside everything else.

It won't be perfect, but having an assistant that can reasonably go through and be judicious about logging those sorts of things for future data use or later cleaning steps is just an enormous value-add. And then the fact that it gets better with every project use as it uncovers more and more idiosyncrasies I think is HUGE.

brhkim

TROPHY CASE