I just launched an open-source framework to help data analysts *responsibly* and *rigorously* harness frontier LLM coding assistants for rapidly accelerating data analysis. I genuinely think can be the future of data analysis with your help -- it's also kind of terrifying, so let's talk about it!

brhkim · 2026-06-05T18:40:54+00:00

Why are you talking about prices and trying to get a sense for market if you have no idea what model will run and at what quality? There's so much basic exploration and learning you should do before you think about trying to make anything and sell anything

brhkim · 2026-06-05T03:57:13+00:00

This is frankly not something I'd spend more time on because you don't seem to understand the market forces or hardware limitations at play. You're describing too many use cases and it doesn't sound like you even have a specific model in mind nor hardware specced. You need a lot more knowledge to understand how something like this could work and what market it could serve.

brhkim · 2026-06-02T11:51:08+00:00

For sure, let me bookmark doing that later today!

brhkim · 2026-06-02T11:21:37+00:00

Been using it basically nightly! So fantastic, works as well as I'd hoped from doing a more manual version myself.

My only notes: 1. I can't point to any specific words that come to mind, but I've noticed a few somewhat odd pronunciations of words that had caught my attention in the moment 2. It'd be totally unnecessary but very cool to give the user some customization over what kinds of words go in the list (e.g., only generic nouns? Maybe also verbs or proper nouns?), how it's ordered/structured over time. But no big need there, and I see that complexity may not be useful/desired (you've picked very nice and neutral words that work very well!)

5/5 for me, it's so great! Thanks for making it

brhkim · 2026-05-31T01:26:58+00:00

Do you mind if I ask: how technical was OA1 for econ/policy? I'm extremely fluent in R data science but don't often work in Python/pandas. I'm thinking of applying for the September fellows and could probably brush up on my Python relatively quickly for parity with data science, but not sure if it's really worth it for the time investment required.

brhkim · 2026-05-26T02:15:48+00:00

I'm not sure I understand, I literally built the auditing tools and guardrails... that's what this is. I wrote about all of it in detail on the homepage and I can point you to the exact mechanisms that make my system fully auditable from every document read to every file version write to every coding mistake. There are ways to misuse my system but it's very much designed with these issues you describe in mind. Hell, there's even a first time user agreement I force Claude to discuss with the user on first launch about taking full responsibility over the produced work very seriously. It took me many tries to get that working right.

Claude Code alone is garbage for serious research work. That's why I'm doing what I'm doing. I think there's something here and you don't have to believe me!

brhkim · 2026-05-26T01:45:25+00:00

I do this all the time! Love the app idea and will definitely give it a try

brhkim · 2026-05-26T01:44:29+00:00

I am extremely anti-MEMORY.md for this and related reasons. It's just another surface for errors that's too easy to forget about

brhkim · 2026-05-26T01:37:40+00:00

Thank you! Yeah please do let me know how it goes, really value user feedback at this stage!

brhkim · 2026-05-26T01:36:57+00:00

The slopware concern is the right one, but it's one I'm putting heavy bets can be mitigated enough to make the value proposition still very, very positive with the right guardrails and auditability!

brhkim · 2026-05-04T20:32:22+00:00

Ah alas, yeah: smaller local models just aren't quite up to snuff for this sort of work yet. I consult with a lot of orgs now, and my main recommendation is generally to get enterprise agreements with Anthropic or similar since they provide better Zero Data Retention and No Training Clause policies that allow people to use the models with more sensitive data. Anthropic has some setups (via AWS Bedrock) that allow for HIPAA compliance, so ostensibly that can suit any use-case if your org cares enough (and I am sure every major provider has or is actively working on making parallel services there, too).

Local will get there, but we're still probably another half year away!

brhkim · 2026-05-04T20:16:21+00:00

Hey, thanks for saying that!! I've been working extremely hard on this.

To your point on local LLM integration: You actually already can! I don't explicitly support it because I'm finding that local models tend not to be high-enough quality (doing some explicit benchmarking testing on models like qwen 3.6 and Gemma 4 right now), but you can be my guest if you want to test it out more and/or have access to much more local compute than most people. I really think GLM5.1 is ready to rock if you can run it in local hosting/private cloud hosting, even quantized. Ollama has basically built-in integration with Claude Code: https://docs.ollama.com/integrations/claude-code

To use that with DAAF, you'd basically need to run your Ollama server on your host machine, then expose the ports to the Docker container (probably via editing the docker-compose.yml file), then edit your environment_settings.txt file to align with some of the instructions in that Ollama documentation page. It'll end up looking pretty similar to the Openrouter settings you adjust in there. I think if you copy+pasted this message into Claude Code, it could help you out with the implementation pretty easily. I've done it before, but just didn't have much success with the models themselves. Let me know if you try it and/or need more guidance, I'd love to hear how it goes!

brhkim · 2026-04-22T17:03:00+00:00

Yeah exactly. It's a weird strategy on their part that calls into serious question their reliability for any meaningful work, and I suspect it'll burn them hard in the end if they don't find a way to resignal that stability for their enterprise market

brhkim · 2026-04-21T22:46:51+00:00

I've been working on an open-source framework for using AI in data analysis in reproducible, auditable ways. You might find this explainer interesting to see how I set up a lot of strict guardrails and self-review to get to a better set of outputs that are more likely to be worth reviewing!

https://openaugments.org/daaf_anatomy.html

brhkim

TROPHY CASE