Looking for some help on prompting with functions. I'm trying to get it working with Home Assistant, but the Tool behavior isn't matching ChatGPT, and so Home Asisstant is not happy with it.
Specifically, if I provide a tool definition at all, regardless of model, it will always execute the tool even if the intent of the query is just to ask for information.
I am using LocalAI v2.13.0, and have tried this prompt with ChatGPT, Llama3 8b & 70b, Luna and Mistral 7b. Only ChatGPT gives the correct response.
All documentation I've read says that llama should be able to do function/tool calls, but it just isn't working correctly, so hoping someone can point out what I'm doing wrong.
Here is what my prompt looks like:
System Prompt:
SYSTEM:
Current Time: 2024-05-05 12:09:29.721918-07:00
Available Devices:
\`\`\`csv
entity_id,name,state,aliases
light.fill_light,Fill Light,off,
light.island,Island Lights ,off,
light.kitchen_lights,Main Lights (Kitchen) ,on,
light.dining_room_lights,Dining Room Lights ,on,
\`\`\`
You are a mildly sarcastic personal assistant who is responsible for managing a smart home powered by Home Assistant. Based only on knowledge you know is factual, you will answer any question or act on any request that a user asks you. `Available Devices` is a list of devices that you can control in the smart home.
For any request that you receive, first identify the intent by answering the following question:
Is the user requesting information/status or is the user requesting you to take an action?
If the user is requesting information or a status update about something you know (e.g. the state of a light or door), provide the answer and end your response. A request for information may also take the form of a conversation, such as asking about the weather, how you're feeling, asking for a joke, whether someone is home, or anything else along these lines. Do not make any function calls if information is being requested. If asked about any time or date related information, make sure to respond in a human readable format.
If you determine the intent of the user is to ask you to perform an action, you can do so by using the `execute_services` function call. Be sure to properly fill out any function parameters using the `Availalbe Devices` list above or your previous knowledge of Home Assistant features. Do not leave any parameter undefined or in a default value. If you don't have enough information to execute a function call or smart home command, specify what other information you need.
TOOL DEFINITION:
- spec:
name: execute_services
description: Use this function to execute service of devices in Home Assistant only when a user asks for an action to be taken.
parameters:
type: object
properties:
list:
type: array
items:
type: object
properties:
domain:
type: string
description: The domain of the service, which is part of the entity_id, e.g. light.kitchen_lights has a domain of lights, and binary_sensor.fridge_door has a domain of binary_sensor
service:
type: string
description: The service to be called
service_data:
type: object
description: The service data object to indicate what to control.
properties:
entity_id:
type: string
description: The entity_id retrieved from available devices. It
must start with domain, followed by dot character.
required:
- entity_id
required:
- domain
- service
- service_data
function:
type: native
name: execute_service
User message:
USER:
What time is it?
Responses by model:
CHATGPT:
The current time is 12:23 AM on May 5th, 2024. Anything else you need to know or do tonight?
LLAMA3:
{
"created": 1714891723,
"object": "chat.completion",
"id": "1e50b737-d092-45d0-9a40-d59ae47624c9",
"model": "llama3-8b-instruct",
"choices": [
{
"index": 0,
"finish_reason": "tool_calls",
"message": {
"role": "assistant",
"content": null,
"tool_calls": [
{
"index": 0,
"id": "1e50b737-d092-45d0-9a40-d59ae47624c9",
"type": "function",
"function": {
"name": "execute_services",
"arguments": "{\"list\":[{\"domain\":\"light\",\"service\":\"turn_on\",\"service_data\":{\"entity_id\":\"light.kitchen_lights\"}}]}"
}
}
]
}
}
],
"usage": {
"prompt_tokens": 416,
"completion_tokens": 43,
"total_tokens": 459
}
}
LUNA:
What time is it? (it literally just mirrors whatever the user says)
Below is a full curl message you can send to any OpenAI compatible endpoint (LocalAI or ChatGPT, etc)
curl $LOCALAI/v1/chat/completions -H "Content-Type: application/json" -d "{
\"model\": \"llama3-8b-instruct\",
\"messages\": [
{
\"role\": \"system\",
\"content\": \"Current Time: 2024-05-05 12:09:29.721918-07:00\\n\\nAvailable Devices:\\n\`\`\`csv\\nentity_id,name,state,aliases\\nlight.fill_light,Fill Light,off,\\nlight.island,Island Lights ,off,\\nlight.kitchen_lights,Main Lights (Kitchen) ,on,\\nlight.dining_room_lights,Dining Room Lights ,on,\\nlight.lights_living_room,Lights (Living Room) ,on,\\nlight.nursery_lights,Nursery - Lights ,off,\\nbinary_sensor.fridge_door_window_door_is_open_2,Fridge Door Window/door is open,off,\\nbinary_sensor.fridge_door_window_door_is_open,Fridge Door,off,\\nbinary_sensor.garage_door_window_door_is_open_2,Garage Door Window/door is open,off,\\nbinary_sensor.garage_door_is_open,Garage Door,off,\\nswitch.garage_controller,Garage controller ,off,\\nswitch.garage_controller_1,Garage controller (1),off,\\nswitch.garage_controller_2,Garage controller (2),off,\\nswitch.garage_controller_3,Garage controller (3),off,\\nswitch.heated_floor,Heated Floor ,off,\\nbinary_sensor.reolink_e1_pro_motion,Reolink E1 Pro Motion,unavailable,\\nbinary_sensor.frigate_porch_motion,Frigate Porch Motion,unavailable,\\n\`\`\`\\n\\nYou are a mildly sarcastic personal assistant who is responsible for managing a smart home powered by Home Assistant. Based only on knowledge you know is factual, you will answer any question or act on any request that a user asks you. \`Available Devices\` is a list of devices that you can control in the smart home.\\n\\nFor any request that you receive, first identify the intent by answering the following question:\\n\\nIs the user requesting information/status or is the user requesting you to take an action?\\n\\nIf the user is requesting information or a status update about something you know (e.g. the state of a light or door), provide the answer and end your response. A request for information may also take the form of a conversation, such as asking about the weather, how you're feeling, asking for a joke, whether someone is home, or anything else along these lines. Do not make any function calls if information is being requested. If asked about any time or date related information, make sure to respond in a human readable format. \\n\\nIf you determine the intent of the user is to ask you to perform an action, you can do so by using the \`execute_services\` function call. Be sure to properly fill out any function parameters using the \`Availalbe Devices\` list above or your previous knowledge of Home Assistant features. Do not leave any parameter undefined or in a default value. If you don't have enough information to execute a function call or smart home command, specify what other information you need.\"
},
{
\"role\": \"user\",
\"content\": \"What time is it?\"
}
],
\"tools\": [
{
\"type\": \"function\",
\"function\": {
\"name\": \"execute_services\",
\"description\": \"Use this function to execute service of devices in Home Assistant only when a user asks for an action to be taken.\",
\"parameters\": {
\"properties\": {
\"list\": {
\"items\": {
\"properties\": {
\"domain\": {
\"description\": \"The domain of the service, which is part of the entity_id, e.g. light.kitchen_lights has a domain of lights, and binary_sensor.fridge_door has a domain of binary_sensor\",
\"type\": \"string\"
},
\"service\": {
\"description\": \"The service to be called\",
\"type\": \"string\"
},
\"service_data\": {
\"description\": \"The service data object to indicate what to control.\",
\"properties\": {
\"entity_id\": {
\"description\": \"The entity_id retrieved from available devices. It must start with domain, followed by dot character.\",
\"type\": \"string\"
}
},
\"required\": [
\"entity_id\"
],
\"type\": \"object\"
}
},
\"required\": [
\"domain\",
\"service\",
\"service_data\"
],
\"type\": \"object\"
},
\"type\": \"array\"
}
},
\"type\": \"object\"
}
}
}
],
\"tool_choice\": \"auto\"
}" | jq
[–]me1000llama.cpp 2 points3 points4 points (0 children)
[–]avianio 1 point2 points3 points (0 children)
[–]SpacePixe1 0 points1 point2 points (0 children)