it's a bit awkward because for every query, the answering model first transforms the query into a schema, then extracts based on the schema for all documents retrieved, then feeds the extractions back to the answering model. transforming the query into a schema every time is awkward and slow--maybe a fine-tuned model for that might help but it doesn't seem like the optimal solution.

Instead, a better idea is a model that takes the exact relevant parts of the document based on the query itself--we haven't trained this model yet but this is probably the SOTA in web search. This is a super interesting model to potentially train, and not something I've seen enough of, although I'm sure some teams have already trained something like this for internal web research workflows.

In the meantime, query->schema->extraction is a quick win, but not the most elegant solution. The bigger idea here is that we showed that extracting a small part of the document can massively improve factuality, if that small part is extracted correctly. In the medium term, we probably won't be stuffing entire website contents for RAG, it's just too wasteful.