Setting Up Chroma

Introduction

Chroma is one of the easiest vector databases to get started with RAG. You can install it locally, it provides a Python API, and works well for prototypes and small to medium projects. This guide walks through installing Chroma, creating a collection, and storing your first documents.

Chroma in a Nutshell

Chroma stores embeddings and metadata in collections. You convert your text into vectors, add them along with metadata, and query them later. Chroma can run in memory, persist to disk, or operate as a Docker container.

Key Terms and Components

Term	Meaning
Collection	Container for related embeddings
Embedding	Vector that represents a text
Metadata	Additional information about a chunk, such as source or category
Persistence	Storing data on disk

When to Use Chroma

Chroma shines when you want to experiment with RAG quickly. It handles many documents, runs locally, and stays simple to use. For large datasets or production systems with high concurrent access, Qdrant or pgvector offer more capabilities.

Installation

Install Chroma easily with pip:

pip install chromadb

A short Python script is all you need for persistent storage in the current directory.

Getting Started

import chromadb

client = chromadb.PersistentClient(path="./chroma_db")

collection = client.create_collection(name="dokumente")

collection.add(
    documents=["RAG verbindet Dokumentensuche mit Sprachmodellen.", "Chroma speichert Vektoren lokal."],
    metadatas=[{"quelle": "artikel"}, {"quelle": "artikel"}],
    ids=["id1", "id2"]
)

results = collection.query(
    query_texts=["Wie funktioniert RAG?"],
    n_results=2
)

print(results)

Chroma generates embeddings automatically. You can also pass your own embedding functions instead.

In-Memory vs. Persistent

Use chromadb.Client() without a path for quick tests. Data vanishes when you exit. For real projects, use PersistentClient or a Docker container to keep your data intact.

Hardware, Cost, and Security Considerations

Chroma is resource-efficient. It uses minimal RAM and disk space. Most applications run fine on standard hardware. It’s open source and free. Since it runs locally, all documents and vectors stay within your own network.

Further AI Resources

• Install Chroma via pip.
• Organize data in collections.
• PersistentClient stores data locally.
• Chroma works well for prototypes and local RAG projects.

Learn more about embeddings in Embedding Models and vector databases in general at Vector Databases.

FAQ - Common Questions About Chroma

Can Chroma work with Ollama?

Yes. Use Ollama for the language model and Chroma for the vector database. Both run locally and communicate through your Python script.

How many documents can Chroma handle?

Chroma works well for many thousands of documents. For millions of entries, switch to more scalable solutions like Qdrant.

Do I need Docker for Chroma?

No, it runs directly via Python. Docker is optional and useful for server setups.

Can I use Chroma with Open WebUI?

Open WebUI supports RAG with various vector databases. Configuration depends on your chosen WebUI version.

Tools and Further Reading

Beyond Chroma, Qdrant, pgvector, and Weaviate are solid alternatives. For getting started, Chroma stands out for simplicity.

Sources

• Chroma Documentation
• Chroma GitHub Repository