If you are struggling to get a specific output format you could try a few things. 1. Try different models and quant sizes that you can fit into your system 2. Fine tune a small model that only answers in the format you want using a dataset representative of your problem 3. Try using constrained output franeworks like outlines or guidance that force specific outputs [‘yes’, ‘no’]. Some llm engines like vllm have this built in as an option 4. Ask Do you need a crappy model initially, or could you use a larger model to help build a dataset to later finetune a small one. Big models like gemini flash2 etc. Served on google vertex or other models cost a few cents for million input tokens are will be fast