System Prompts for Ollama

What this article covers

• What a system prompt is and how it works.
• How to use system prompts in Ollama.
• Examples for different roles and formats.
• How to structure outputs and avoid common mistakes.
• The difference between system, user, and assistant prompts.

Introduction: System Prompts for Ollama

A system prompt defines how a language model should behave: which role it takes on, what rules apply, and what responses should look like. Unlike regular user prompts, the system prompt remains active throughout the entire conversation. In Ollama, you can pass it via the API, in Modelfiles, or through model configuration. When you use system prompts strategically, you get more consistent and useful answers.

This article shows you how system prompts are structured and which patterns work well.

Key terms

• System prompt: A controlling instruction at the system level.
• User prompt: A normal user question or task.
• Assistant: The model’s response.
• Role: The character or task of the model.
• Rule: A constraint or instruction for behavior.
• Format: The structure of the output.
• Token: The smallest unit the model processes.

Structure of a system prompt

A good system prompt contains:

• Role: What is the model?
• Goal: What should it achieve?
• Rules: What is allowed and what is not?
• Format: How should responses look?
• Examples: Optional, but helpful.

Example

You are a technical advisor for local AI. Answer questions precisely and in English. If information is unknown, say so honestly. Structure answers with headings and lists.

System prompt in the API

curl http://localhost:11434/api/chat -d '{
  "model": "llama3.1",
  "messages": [
    {"role": "system", "content": "You are a friendly assistant."},
    {"role": "user", "content": "Hello"}
  ]
}'

System prompt in the Modelfile

FROM llama3.1

SYSTEM """
You are a technical advisor for local AI. Answer precisely and with examples.
"""

PARAMETER temperature 0.3

Good and bad system prompts

Good

• Short and specific.
• Clear roles and rules.
• Simple language.
• No unnecessary details.

Bad

• Too long.
• Vague wording.
• Contradictory rules.
• Too many nested conditions.
• Wastes context tokens unnecessarily.

Examples for different applications

Coding assistant

You are an experienced Python developer. Write clean, well-commented code. Briefly explain what the code does. Point out potential error sources.

RAG assistant

You answer questions exclusively based on the provided context. If the answer is not in the context, indicate that no relevant information is available. Quote relevant passages.

Translator

You are a professional translator. Translate texts with stylistic precision and semantic accuracy. Provide no explanations unless asked.

Teacher

You are a patient teacher. Explain concepts simply with everyday examples. Ask a short comprehension question at the end.

Safe assistant

Answer only questions that rest on documented facts. Do not speculate or make sensitive claims. Refer to official documentation.

System prompt and temperature

A precise system prompt works best with low temperature:

PARAMETER temperature 0.1
PARAMETER top_p 0.9

For creative tasks, temperature can be higher.

Observing model behavior

Not all models follow system prompts equally well. Some models tend to ignore system instructions or incorporate them into responses. What matters:

• Test multiple models.
• Check behavior over long conversations.
• Repeat formatting rules regularly.
• For critical formats, enforce JSON or Markdown.

Dynamic system prompts

In applications, the system prompt can be adjusted based on context:

system_prompt = "You are a network expert." if topic == "network" else "You are a coding expert."

This keeps behavior flexible.

Common pitfalls

• System prompt too long: Consumes valuable context.
• Contradictions: Model chooses arbitrarily.
• No format specifications: Responses are unstructured.
• Ignored prompting: Some models follow system prompts poorly.
• Wrong order: System prompt must come first.
• User overrides: Some applications let users change the system prompt, creating security risks.

Further reading and resources

• BotServ.de Ollama Modelfiles
• BotServ.de Ollama Prompt Engineering
• BotServ.de Ollama for Coding
• BotServ.de RAG with Ollama

FAQ: System Prompts for Ollama

How important is the system prompt? Very important, especially with smaller models.

Can I use multiple system prompts? Technically yes, but in practice only one should be active per conversation.

Is the system prompt visible in the output? No, it only controls behavior.

Do I need a system prompt for simple questions? Not always, but it ensures more consistent answers.

Can the user override the system prompt? In pure Ollama API applications, yes, unless implemented otherwise.

Sources and further reading

• Ollama Modelfile Docs: https://github.com/ollama/ollama/blob/main/docs/modelfile.md
• Prompting Guide: https://www.promptingguide.ai/
• System Messages Best Practices: https://platform.openai.com/docs/guides/prompt-engineering

Summary: System Prompts for Ollama

System prompts are a powerful tool for controlling Ollama models deliberately. They define role, rules, and output format for the entire conversation. Short, precise, and consistent system prompts work best, especially combined with low temperature and appropriate model parameters. When you use system prompts consistently in Modelfiles or the API, you get significantly more predictable and useful answers.