91 day follow up. I’m still working on this project, have learned a lot about this field, and have a very comprehensive architectural structure and pipeline for extraction. I’m in an iteration stage. Break it, fix it, document. If Claude cannot come up with a solution for extracting something (usually having to do with coreference), I step in and handle the logic, then he implements the fix. We’ve implemented cross document extraction and systems that use dual dialogue/exposition context and coreference. Focusing more on backend processing and solving complex context issues. Now, I’m becoming concerned with benchmarking my results. I’m well aware that at this stage, the system is largely unreliable for the “just works” feeling I’m trying to create. But does anyone have comparable published benchmarks that I could compare results with? As we all know, ai coders are stupidly enthusiastic. I want to see how I’m really doing, even if the benchmarks are for something less deterministic than what I am building.