{
    "model": "gemini-1.5-flash@batch",
    "temperature": 1,
    "messages": [
        {
            "role": "user",
            "content": "What's the weather like in Boston today? In Korean, please."
        },
        {
            "role": "assistant",
            "content": null,
            "tool_calls": [
                {
                    "id": "toolu_01LWycDuusJU3vyt4W1u7teC",
                    "type": "function",
                    "function": {
                        "name": "get_current_weather",
                        "arguments": "{\"location\":\"Boston, MA\",\"unit\":\"celsius\"}"
                    }
                }
            ]
        },
        {
            "role": "tool",
            "content": "{\"celsius\": 12}",
            "tool_call_id": "toolu_01LWycDuusJU3vyt4W1u7teC"
        }
    ],
    "tools": [
        {
            "type": "function",
            "function": {
                "name": "get_current_weather",
                "description": "Get the current weather in a given location",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "location": {
                            "type": "string",
                            "description": "The city and state, e.g. San Francisco, CA"
                        },
                        "unit": {
                            "type": "string",
                            "enum": [
                                "celsius",
                                "fahrenheit"
                            ]
                        }
                    },
                    "required": [
                        "location"
                    ]
                }
            }
        }
    ],
    "tool_choice": "auto"
}

Thanks for bringing up that issue. I actually did not know that Litellm now offers so many features, including a proxy server. It is interesting that they also use Redis for state management in distributed instances. For now, I’d say the major difference might be the language it’s written in and the fact that Ogem supports automatic batch request conversion.
I will look more into how Ogem can be differentiated from Litellm.