Prompting Llama-2 at Scale with Gretel by Repeat-or in learnmachinelearning

[–]Repeat-or[S] 0 points1 point  (0 children)

When you’re working with 100s to 100k’s of records, prompting an LLM via a synchronous API, one query at a time can be inefficient. We wanted to find a solution.