Ollama Function Calling

What This Article Covers

• What function calling is.
• How to define tools in Ollama.
• JSON mode and force-output.
• Which models support functions.
• Practical Python example.

Introduction: Ollama Function Calling

Function calling allows language models to trigger specific actions by returning structured parameters for external functions. Instead of formulating the answer itself, the model invokes a tool function, such as fetching weather data, performing a calculation, or searching for information. In Ollama, function calling has limited availability and typically requires JSON output and a model trained to understand tools.

This article shows how function calling works with Ollama.

Key Terms

• Function Calling: Model invokes external functions.
• Tool: External function known to the model.
• Schema: JSON schema for function parameters.
• JSON Mode: Forces valid JSON output.
• force: Compels the model to call a specific function.
• OpenAI Tools: Industry standard for function calling.
• Ollama tools: Ollama-specific implementation.
• Stop/Start: Control characters for JSON.

Support in Ollama

Function calling in Ollama is newer and less comprehensive than with OpenAI. Some models like Qwen 2.5, Llama 3.1, and certain Mistral variants can use tools. The API follows the OpenAI tool format.

Defining a Tool

{
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Returns current weather for a location.",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "City or location"
            },
            "unit": {
              "type": "string",
              "enum": ["celsius", "fahrenheit"]
            }
          },
          "required": ["location"]
        }
      }
    }
  ]
}

Python Example

import requests

url = "http://localhost:11434/api/chat"
payload = {
    "model": "qwen2.5:7b",
    "messages": [
        {"role": "user", "content": "What's the weather in Berlin?"}
    ],
    "tools": [
        {
            "type": "function",
            "function": {
                "name": "get_weather",
                "description": "Returns current weather.",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "location": {"type": "string"},
                        "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
                    },
                    "required": ["location"]
                }
            }
        }
    ],
    "stream": False
}

response = requests.post(url, json=payload)
print(response.json())

Evaluating the Result

The model responds with a tool_calls structure:

{
  "message": {
    "role": "assistant",
    "tool_calls": [
      {
        "function": {
          "name": "get_weather",
          "arguments": {"location": "Berlin", "unit": "celsius"}
        }
      }
    ]
  }
}

Executing the Function

Your application executes the function and adds the result back to the conversation history:

tool_result = get_weather("Berlin", "celsius")

payload["messages"].append({
    "role": "tool",
    "content": tool_result,
    "name": "get_weather"
})

response = requests.post(url, json=payload)
print(response.json()["message"]["content"])

Force JSON Mode

{
  "format": "json"
}

This helps if the model otherwise wraps JSON output in markdown blocks.

Ollama Modelfile with Tools

FROM qwen2.5:7b

PARAMETER temperature 0.1
SYSTEM "You are a helper who uses tools to complete tasks."

Not all parameters directly define tools. They are passed via the API instead.

Tips

• Choose a model that supports tools.
• Keep schemas clear and simple.
• Use descriptive tool names.
• Always validate responses.
• Handle errors in case the model hallucinates.
• Use few tools per request.
• Specify parameter types.

Common Pitfalls

• Model doesn’t support tools: Smaller or older models ignore tools.
• No tool invocation: Model responds normally instead of calling.
• JSON errors: Output must be parseable.
• Wrong parameters: Schema is too complex.
• Multiple tool calls: Your application must handle several calls sequentially.
• CORS/Network issues: Client cannot reach the API.

Further Reading and Resources

• BotServ.de Ollama Model Gallery
• BotServ.de Ollama API Troubleshooting
• BotServ.de Ollama in Docker
• BotServ.de AI Agent Frameworks

FAQ: Ollama Function Calling

Which models support function calling? Qwen 2.5, Llama 3.1, and some Mistral variants.

Do I need to define tools in the Modelfile? No, they are passed via the API.

Can I use multiple tools at once? Yes, depending on the model and query.

How do I force JSON output? Use "format": "json".

Does function calling work with all models? No, only models trained for it.

Sources and Further Reading

• Ollama Tools: https://ollama.com/blog/tool-support
• OpenAI Function Calling: https://platform.openai.com/docs/guides/function-calling

Summary: Ollama Function Calling

Function calling enables Ollama models to invoke external tools. Tools are defined as JSON schemas and passed via the API. Models like Qwen 2.5 and Llama 3.1 support this mode. Your application receives tool calls, executes the functions, and returns results to the model. With clear schemas, JSON output, and proper validation, you can build agent workflows even locally.