Ollama Modelfiles

What this article covers

• What a Modelfile is and why you need it.
• Creating custom models with system prompts.
• Key parameters like temperature and context length.
• Integrating .gguf models and adapters.
• Practical examples for chat, coding, and RAG.

Introduction: Ollama Modelfiles

Ollama Modelfiles are simple text files that describe how a model should be loaded and behave. They let you build custom assistants, specialized chatbots, or coding models. A Modelfile specifies the base model, defines system prompts, configures parameters, and can even include additional files like LoRA adapters.

This article walks through the structure and provides examples of typical Modelfiles.

Key concepts

• Modelfile: A blueprint for an Ollama model.
• FROM: The base model that serves as the foundation.
• SYSTEM: A system prompt that controls behavior.
• PARAMETER: Settings like temperature or context length.
• MESSAGE: Example conversation for few-shot learning.
• ADAPTER: LoRA or other fine-tuning layers.
• LICENSE: License information.

Basic structure

FROM llama3.1

SYSTEM """
You are a friendly assistant for local AI. Respond concisely and with clarity.
"""

PARAMETER temperature 0.7
PARAMETER num_ctx 4096

To create and run it:

ollama create my-assistant -f Modelfile
ollama run my-assistant

Key parameters

Parameter	Description
temperature	Creativity level, 0 to 1.
top_p	Nucleus sampling, 0 to 1.
top_k	Number of top-K tokens.
num_predict	Maximum number of tokens to generate.
num_ctx	Context length.
repeat_penalty	Penalty for repeated tokens.
seed	Random seed for reproducibility.

Example:

PARAMETER temperature 0.5
PARAMETER top_p 0.9
PARAMETER num_ctx 8192
PARAMETER repeat_penalty 1.2

System prompts

The system prompt defines the model’s role and rules. It has a significant impact on response quality.

SYSTEM """
You are an expert AI consultant. Answer briefly, with precision and concrete examples.
"""

Few-shot examples

Use MESSAGE to set example conversations:

MESSAGE user "What is Docker?"
MESSAGE assistant "Docker is a platform for running applications in containers."
MESSAGE user "How do I start a container?"
MESSAGE assistant "Use the command `docker run <image>`."

This helps the model learn the desired response format.

Copying and modifying models

ollama cp llama3.1 my-llama

Then create a Modelfile for my-llama and build a new model from it.

Using custom GGUF models

For self-quantized or Hugging Face models:

FROM ./my-model.q4_K_M.gguf

SYSTEM """
Custom model, running locally.
"""

Adding LoRA adapters

FROM llama3.1
ADAPTER ./my-lora-adapter.bin

The adapter fine-tunes behavior for specific tasks.

Example: coding assistant

FROM qwen2.5-coder:14b

SYSTEM """
You are an experienced software developer. Write clean, commented code and briefly explain what it does.
"""

PARAMETER temperature 0.2
PARAMETER num_ctx 8192

Example: RAG assistant

FROM llama3.1

SYSTEM """
Answer questions solely based on the provided context. If the answer is not in the context, say so honestly.
"""

PARAMETER temperature 0.1
PARAMETER num_ctx 8192

Example: creative writing

FROM llama3.1

SYSTEM """
You are a creative writing assistant. Produce engaging text with a distinctive voice.
"""

PARAMETER temperature 0.8
PARAMETER top_p 0.95

Tips

• Keep system prompts short and precise.
• Adjust temperature based on the task.
• Don’t set context length unnecessarily high.
• Provide examples when a specific response format is needed.
• Test results and iterate.

Common pitfalls

• Wrong format: Modelfile syntax must be correct.
• Base model unavailable: Run ollama pull first.
• Context too large: Slows down or causes crashes.
• Wrong parameters: num_ctx is sometimes ignored if the model doesn’t support it.
• System prompt too long: Consumes valuable context.
• Incompatible adapter: LoRA must match the base model.

Further reading

• BotServ.de Ollama commands
• BotServ.de Ollama model lifecycle
• BotServ.de Ollama performance
• BotServ.de Ollama for coding
• BotServ.de RAG knowledge base

FAQ: Ollama Modelfiles

Do I need a Modelfile? Only if you want to adjust a model’s behavior or parameters.

Can I set multiple parameters? Yes, add as many PARAMETER lines as needed.

Where do I save the Modelfile? Anywhere on your filesystem, for example in your project directory.

How do I delete a created model? Use ollama rm <name>.

Can I share the model? Yes, by exporting the model or documenting the Modelfile.

Sources and further reading

• Ollama Modelfile Docs: https://github.com/ollama/ollama/blob/main/docs/modelfile.md
• Ollama Library: https://ollama.com/library
• GGUF Format: https://github.com/ggerganov/ggml/blob/master/docs/gguf.md

Summary: Ollama Modelfiles

Modelfiles are the simplest way to define custom Ollama models. They specify the base model, system prompt, parameters, and examples. With just a few lines, you can build a specialized assistant for coding, RAG, or creative writing. The key is precise prompts, appropriate parameters, and thorough testing. Master Modelfiles and you’ll use Ollama far more effectively for your own applications.