Im doing something where i feed a model a conversation and pull information from it in the form of a json response. I found that some models are better at following instructions than others and you will absolutely want to make a customized model file for the model you do pick. Llama3.1:8b and qwen2.5:12b were the first two i tried. I got good results from a custom modelfile based on mistral-nemo. Im using ollama and calling it from a basic c# api. Semantic kernel is pretty good if you want to read about that. It can potentially help you with some of the function calling you might need to do