Using MCP with Ollama

What this article covers

• What the Model Context Protocol is.
• How tools and context are provided via MCP.
• How Ollama can be used with MCP-like workflows.
• Examples of tools, system prompts, and function calls.
• Tips for local implementations.

Introduction: Using MCP with Ollama

MCP stands for Model Context Protocol. It’s an open standard that lets language models use external tools, data sources, and contexts in a structured way. Rather than cramming everything into a prompt, models can invoke functions on demand, process results, and tackle more complex tasks. With Ollama, you can build similar behavior by embedding tools in your API request and interpreting the model’s responses.

This article walks through the idea behind MCP, how tools are defined, and how to implement a simple example with Ollama.

Key concepts

• MCP: Model Context Protocol.
• Tool: External function the model can call.
• Function Calling: Model decides which function to invoke.
• Schema: Description of a function and its parameters.
• Context: Additional information available to the model.
• Agent: System that combines model, tools, and logic.
• Server: Component that provides tools.
• Client: Application that sends requests to the MCP server and model.

Why use MCP?

MCP helps you:

• Supply models with current data.
• Access local files, databases, or APIs.
• Execute calculations or actions.
• Automate repetitive tasks.
• Enable multi-step workflows.

Instead of a model memorizing everything, it fetches information from its environment as needed.

Defining tools

A tool is described by a JSON schema. Here’s an example weather function:

{
  "type": "function",
  "function": {
    "name": "get_weather",
    "description": "Returns the current weather for a location.",
    "parameters": {
      "type": "object",
      "properties": {
        "location": {
          "type": "string",
          "description": "City or place"
        }
      },
      "required": ["location"]
    }
  }
}

Ollama and function calling

Ollama doesn’t offer a native MCP server, but it supports embedding tools via the chat format. Certain models return a special response format that you can interpret as a function call.

Example with Python

import ollama
import json

weather_schema = {
    "name": "get_weather",
    "description": "Returns the current weather for a location.",
    "parameters": {
        "type": "object",
        "properties": {
            "location": {"type": "string"}
        },
        "required": ["location"]
    }
}

def get_weather(location):
    return f"Weather in {location} is sunny and 22 degrees."

messages = [
    {"role": "user", "content": "What's the weather in Berlin?"}
]

response = ollama.chat(
    model="qwen2.5:14b",
    messages=messages,
    tools=[weather_schema]
)

print(response["message"]["content"])

When the model encodes a tool call in its response, your application can execute the function and send the result back.

Two-step workflow

import re

answer = response["message"]["content"]
match = re.search(r"get_weather\(.*location=[\"'](.+?)[\"']\)", answer)

if match:
    location = match.group(1)
    result = get_weather(location)

    messages.append({"role": "assistant", "content": answer})
    messages.append({"role": "tool", "content": result})

    final = ollama.chat(model="qwen2.5:14b", messages=messages)
    print(final["message"]["content"])

Important: The tool call format depends on the model. Specialized tool models deliver more structured responses.

Building an MCP server

An MCP server provides tools and communicates via a standardized protocol. For Ollama, you can write a simple Python server that exchanges JSON schemas and results over HTTP:

from fastapi import FastAPI

app = FastAPI()

@app.get("/tools")
def tools():
    return [weather_schema]

@app.post("/call")
def call(payload: dict):
    if payload["name"] == "get_weather":
        return get_weather(payload["args"]["location"])

Practical applications

• RAG: Model retrieves relevant document sections.
• File search: Model searches a directory.
• Calendar: Model reads or writes appointments.
• System information: Model queries CPU or RAM usage.
• API access: Model fetches data from local or external APIs.

Tips

• Use clear schemas with good descriptions.
• A few precise tools yield better results than many poorly documented ones.
• Validate tool calls before executing them.
• Mind security: never run unvalidated commands.
• Catch errors and pass them back to the model as context.
• Keep logs to trace workflows.

Common pitfalls

• Model doesn’t support function calling: Output isn’t structured.
• Wrong format: Model invents its own function syntax.
• Too many tools: Model gets overwhelmed.
• Missing description: Model picks the wrong tool.
• Security gaps: Tool executes arbitrary commands.
• Circular calls: Model calls tools endlessly.
• No timeout logic: External calls block the workflow.

Further reading and resources

• BotServ.de Ollama API libraries
• BotServ.de Ollama for coding
• BotServ.de RAG with Ollama
• BotServ.de Agent memory

FAQ: MCP with Ollama

Does Ollama support MCP natively? Not as a server, but tools can be embedded via the chat format.

Which models are good for function calling? qwen2.5, mistral, and models trained specifically for tools often deliver usable results.

Do I need an MCP server? Only if multiple applications should use the same tools. A simple Python loop is often enough.

Is MCP secure? The protocol is as secure as the tools you implement. Always validate inputs.

Can I use MCP with Open WebUI? Open WebUI has its own tool features. Direct MCP integration may vary by version.

Sources and further reading

• MCP Spec: https://modelcontextprotocol.io/
• Ollama API: https://github.com/ollama/ollama/blob/main/docs/api.md
• LangChain Tools: https://python.langchain.com/docs/modules/agents/tools/

Summary: Using MCP with Ollama

MCP extends local language models with tools, data sources, and dynamic context. Ollama enables similar workflows through function-calling-like prompts and interpretation of model responses. Implement clear schemas, secure execution logic, and a thoughtful two-step workflow, and you can turn Ollama into a privacy-friendly agent with tool access. Especially powerful when combined with RAG, APIs, and local system data.