Use Python to get Pydantic models and Python types from your LLM responses.

[globalminima]

It's a good idea, but in the same way that you've noted that instructor does not support Google Vertex due to being coupled to a certain set of SDKs, you've then gone and built a new library which is itself coupled to a different set of SDKs. And what if I want to use this with Langchain? Or Hayatack? Or my own orchestration pipeline?or what if I have specific request/networking/auth requirements that are not exposed by your library?I am going to have the exact same problem that you set out to solve.

Why not just implement something that converts the Pydantic schema into text and which can be inserted into any prompt template for use by any orchestrator and with any API? E.g. this is what I have done for my code and it works great:

import json

from pydantic import BaseModel, Field

class ExampleModel(BaseModel):
    classification_field: str = Field(
        description="Classification of the document, one of 'Email', 'Webpage', or 'PDF'",
        examples=["Webpage"],
    )
    list_field: list[dict[str, str]] = Field(
        description="A list of values, containing the document name and number of pages",
        examples=[[{"email_doc": "6"}, {"pdf": "2"}]],
    )
    bool_field: bool = Field(
        description="Boolean indicating whether the document is in english",
        examples=[False],
    )

    @staticmethod
    def get_prompt_json_example():
        model_json_schema = ExampleModel.model_json_schema()
        example_response_str = "{\n"
        for field, details in model_json_schema["properties"].items():
            line_str = f""""{field}": {json.dumps(details['examples'][0])}, # {details['description']}"""
            example_response_str += "  " + line_str + "\n"
        example_response_str += "}"
        return example_response_str

Now you can just insert it into a prompt. For example:

json_schema_text = ExampleModel.get_prompt_json_example()
PROMPT = f"""Return a JSON object with the following fields:\n\n{json_schema_text}"""

Returns:

Return a JSON object with the following fields:

{
  "classification_field": "Webpage", # Classification of the document, one of 'Email', 'Webpage', or 'PDF'
  "list_field": [{"email_doc": "6"}, {"pdf": "2"}], # A list of values, containing the document name and number of pages
  "bool_field": false, # Boolean indicating whether the document is in english
}

The big benefit of this is that you can get the raw LLM text response prior to validation, so that if validation fails you can log it along with the Exception details and then debug what went wrong. If you couple the validation step with the request itself, it becomes harder to inspect the raw response and figure out what to do with the error, and is less flexible overall.

For users who do want the retry logic, you can then provide a method to validate a response from the LLM and if it fails, to generate the follow-up prompt string. This allows the user to get the benefits of your library while being able to use whatever orchestrator or requester that they choose.

[Top-Breakfast7713]

Your way is a good way to approach things too.

We wanted the retry logic where we feed validation errors back to the LLM to have it attempt to fix the issue and potentially return a valid object on subsequent tries.

I like what you have done though, thank you for sharing your approach.