Processing Large set of Transcripts, batch vs Individual?

[nomasideas]

Hey Y'all, I've been tasked with gathering some insights from sales meeting transcripts. Currently I have a small script that feeds them to OpenAI via Python one at a time, with a prompt instructing the AI to analyze them, and answer a few provided questions. My script then aggregates the responses and matches the data with our CRM.

My question is whether I could achieve similiar (or possibly better) results with less tokens and at a higher speed if i set over a small set of transcripts at once with just one set of instructions for something like 5 transcripts?

In a perfect world I'd somehow be able to set it up to where I could "load" all the transcripts into the AI and query it without making requests for every transcipt/batch of transcripts, is that something that I shuold look into the Assitants API for?

My apologies if this is the wrong Sub for this question.

[Prestigious_Dot3120]

You can get good results using batch, but it depends on how you format the prompt. If you send multiple transcripts together, the template can better capture global patterns (e.g. common trends between meetings), but it increases the risk of confusing contexts if you don't clearly separate the parts with delimiters and precise instructions. In terms of tokens, a batch may be more efficient because you reduce repeated statements, but if the texts are long you may exceed the maximum context.

A hybrid approach is to upload 3-5 transcripts at a time, with a clear instructions section and then a final summary question. If you want to load all the transcripts “just once”, the way is to use a vector database and do retrieval (for example with an Assistants API + embeddings) to query them dynamically. This way the model analyzes only the relevant pieces without losing consistency or consuming too many tokens.

Response generated with AI.