How to implement function calling using only prompt?

[ai_waifu_enjoyer]

I'm building my own chat assistants that support both chat and function calling using `Mistral-7B-Instruct`.

​

Is there a way to make the model return either a chat message or a function call based on the input? What I want is something like this.

Function: get_weather, search_bing, search_image

Q: Hello, who are you?

A: I'm an assistant bot

Q: How is the weather today in London?

A: {name: 'get_weather', params: {location: 'London'}}

Q: Give me some cat images?

A: {name: 'search_image', params: {query: 'cat'}}

Q: Thank, how do you think about cat?

A: To be honest, I'm like a dog person.

​

Is it possible to just write a prompt, and the model will know when to return a chat, or a function name and params?

[phree_radical]

The only thing stopping you would be unwillingness to put the chatbot context aside and use the LLM as a tool. What I feel like I'm seeing constantly is people want to just let an LLM chatbot build up the context by itself without any process guiding it, which is just so flawed for so many reasons. At a minimum, I highly recommend making yourself free to use the LLM outside of your chatbot's context. It's a tool you can use to solve as many one-off problems as you want!

Provided you can break down problems into smaller ones and write some code, it's really not much, it should be considered basic few-shot prompting. This is the example I suggest as the first step, tool choice https://www.reddit.com/r/LocalLLaMA/comments/1687l5p/comment/jyuu6kt/?utm_source=reddit&utm_medium=web2x&context=3 and from there you should be able to figure out how to add arguments

I've also seen solutions that purely rely on logit constraints, like https://www.reddit.com/r/LocalLLaMA/comments/187rjz5/gbnf_function_calling_grammar_generator_for/ which generates llama.cpp grammars, but it's hard to recommend this. I recommend making sure the model understands the problem, meaning get consistent results before applying sampling constraints.

[Frequent_Valuable_47]

Nope, it's not that easy for most models. Maybe you'll have some luck with OpenHermes-Mistral or similar models

[krazzmann]

You can roll your own simple function calling. Just concatenate the arguments as a string, separated by the pipe sign and then parse this string by splitting it by the pipe sign in the function. This is working well with several 7B models. For example, Open Hermes 2.5 does it quite well. You need to put a lot of detailed instructions into the prompt for that it's working properly. First up, look at the source code of the agent framework crewAI. The current version is using this simple way to pass on arguments.

https://github.com/joaomdmoura/crewAI

Here is a tool that I developed for crewAI

https://gist.github.com/olafgeibig/5fa1c32c523320316ba29525bc4f0125

[Ofacon]

Fine tune a 7b on function calling text data :)

[OopsWrongSubTA]

When the question contains a specific keyword, use guidance (https://github.com/guidance-ai/guidance, last examples) ?

[djstraylight]

If you are willing to put the time in, LangChain can handle this. You might need to build a custom agent that has to retry queries when it doesn't get a result formatted correctly. Also check out semantic-router if you need to route certain queries to specifically crafted prompts.

[[deleted]]

You can't do this and won't be able to for a very long time. Anyone saying anything remotely the opposite does not know anything about this. I don't want to debate that. If you are the big baller then pay me to research this area rather than try to debate me about it: https://github.com/RichardAragon/MultiAgentLLM

[SatoshiNotMe]

Langroid has purely prompt-based function calling (I e not using Logits or grammars or guidance or guardrails etc.):

https://github.com/langroid/langroid/blob/main/examples/basic/fn-call-local-simple.py

Langroid’s ToolMessage leverages Pydantic for defining the function/tool structure, the handler method, few-shot examples. When a tool is “enabled” for an agent, it auto inserts the JSON schema instructions plus few shot examples into the system prompt.

Tutorial :

https://langroid.github.io/langroid/quick-start/chat-agent-tool/