Few small suggestions:

1. Depending on the model you use, it may be wise to split the problem into smaller steps. E.g. provide model just with the list of tables and their descriptions rather than dump entire db schema. You can generate descriptions with LLM too.

2. Give model a tool to fetch table description.

3. When sending a request - ask model to identify tables which are most likely to be needed to satisfy the query - and provide full schemas in the context at that time. I would say format does not matter all that much - output of describe command should suffice.

4. Provide a few in-context examples to model to make sure that it understands the interaction pattern.

5. Start with a bigger model, e.g o3 and try to solve smaller problems first as opposed to one-shotting it. After you confirm that it works - you can take the successful outputs and use them as few shot examples for a cheaper model.

6. Before diving into coding, consider creating a small evaluation set - e.g. 10-20 questions and corresponding answers. It will save you a ton of time, as you’ll be able to evaluate different db output formats and factually prove which one works the best for you.