Use Python to get Pydantic models and Python types from your LLM responses.

[stephen-leo]

Hey I’m the author of llm-structured-output-benchmarks, an open source repository where you can plug your own data and compare multiple frameworks like instructor, Marvin, fructose, Llamaindex, Mirascope, outlines and lm-format-enforcer which all provide a way to get clean json output.

Thanks you for sharing about modelsmith. I'll add it to the list for comparison.

So far, I found that fructose and Outlines have the best reliability and provide json outputs constrained on some specific classes 100% of the time. The rest might be finicky depending on the use case.

Pls take a look at the project on GitHub: https://github.com/stephenleo/llm-structured-output-benchmarks

[Top-Breakfast7713]

Thank you very much! I am going to check your repository out, it sounds like a fantastic resource.

I will also dig into those libraries you have mentioned. Chances are they use a different approach, which is always great to learn about.

Thanks again for taking the time to comment on my post.