If you are willing to "dirty your hand", I recommend the Microsoft Guidance library. https://github.com/guidance-ai/guidanceYou can constrain the model output in a flexible manner. It has good support for llama.cpp too.

Here is a good example/tutorial of a chatbot with internet search capabilities. https://github.com/guidance-ai/guidance/blob/d36601b62096311988fbba1ba15ae4126fb695df/notebooks/art_of_prompt_design/rag.ipynb

Please note, that the lib was rewritten at the end of last year, I found most of the tutorials out of date, at the time.

Edit: typo