Integrating Home Assistant with Ollama

IRC-Mania Die Seite rund um Sicherheit und Programmierung

Internet, Sicherheit , Internet Relay Chat

Gesponsert

What this article covers

• Setting up Ollama as an AI backend for Home Assistant.
• Using a local voice assistant with Ollama.
• Building automations that make AI-driven decisions.
• Practical examples of natural language commands and intelligent control.
• Best practices for performance and privacy.

Introduction: Home Assistant with Ollama explained

Home Assistant includes a built-in voice assistant called Assist. By default, it relies on cloud services. With Ollama as your backend, everything stays local: you speak, Whisper transcribes, Ollama understands and decides, Home Assistant executes. No data leaves your network.

This article is for users who want to connect Home Assistant to local AI. Background information is available in Home Assistant and Ollama.

Why use Home Assistant with Ollama?

Commercial voice assistants (Alexa, Google) send your speech commands to the cloud. Home Assistant with Ollama processes everything locally: “Turn on the living room lights” gets transcribed by Whisper, interpreted by Ollama, and executed by Home Assistant without a single byte leaving your home.

How Home Assistant with Ollama works

Ollama integrates into Home Assistant as a Conversation agent. Assist (the voice assistant) then uses Ollama instead of cloud services. Beyond that, you can leverage Ollama in automations and scripts for intelligent decision-making.

The core idea is simple: local AI as the brain for your smart home.

Who should read this?

• Home Assistant users who want local AI.
• Privacy-conscious people avoiding cloud voice assistants.
• Self-hosters running Ollama in their smart homes.
• Tinkerers building natural language control systems.

Familiarity with Home Assistant and Ollama is helpful.

Key terms

• Home Assistant - Smart home platform. Use when: the foundation.
• Assist - HA voice assistant. Use when: you need voice control.
• Ollama - Local model server. Use when: you need the AI backend.
• Whisper - Speech-to-text. Use when: handling voice input.
• Piper - Text-to-speech. Use when: generating voice output.
• Conversation Agent - AI backend for Assist. Use when: connecting the pieces.
• Wyoming - Protocol for voice services. Use when: running Whisper or Piper.

Setup: Ollama as a Conversation Agent

1. Ollama is running

# Ollama running on http://ollama:11434
# Load a model:
ollama pull llama3.1

2. Ollama integration in HA

# configuration.yaml
ollama:
  url: http://ollama:11434

Or via the UI:

1. Settings → Devices & Services → Add Integration
2. Search for “Ollama”
3. Enter URL: http://ollama:11434
4. Select model: llama3.1

3. Configure Assist

# configuration.yaml
conversation:
  intents:
    # Custom intents here

Via the UI:

1. Settings → Voice Assistants → Assist
2. Conversation Agent: Ollama
3. STT: Whisper (local)
4. TTS: Piper (local)

Fully local voice assistant

Voice input
    │
    ▼
Whisper (STT) ──► Text
    │
    ▼
Ollama (LLM) ──► Understands command
    │
    ▼
Home Assistant ──► Executes
    │
    ▼
Piper (TTS) ──► Voice output

Wyoming containers for Whisper and Piper

version: "3.8"

services:
  whisper:
    image: rhasspy/wyoming-whisper:latest
    container_name: whisper
    restart: unless-stopped
    ports:
      - "10300:10300"
    volumes:
      - whisper_data:/data
    command: --model small --language de
    networks:
      - smarthome

  piper:
    image: rhasspy/wyoming-piper:latest
    container_name: piper
    restart: unless-stopped
    ports:
      - "10200:10200"
    volumes:
      - piper_data:/data
    command: --voice de_DE-thorsten-high
    networks:
      - smarthome

  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    restart: unless-stopped
    ports:
      - "11434:11434"
    volumes:
      - ollama_data:/root/.ollama
    networks:
      - smarthome

  homeassistant:
    image: homeassistant/home-assistant:latest
    container_name: homeassistant
    restart: unless-stopped
    ports:
      - "8123:8123"
    volumes:
      - ha_config:/config
    networks:
      - smarthome

volumes:
  whisper_data:
  piper_data:
  ollama_data:
  ha_config:

networks:
  smarthome:
    driver: bridge

Practical example 1: Natural language commands

# Home Assistant automation
automation:
  - alias: "AI light control"
    trigger:
      - platform: conversation
        command:
          - "Turn [the] lights [in] {room} [on|off]"
    action:
      - service: light.toggle
        target:
          area_id: "{{ trigger.slots.room }}"

Or more complex with Ollama:

automation:
  - alias: "Interpret AI command"
    trigger:
      - platform: event
        event_type: voice_command
    action:
      - service: ollama.generate
        data:
          prompt: "Interpret: '{{ trigger.event.data.command }}'
                   Available actions: lights_on, lights_off, temperature,
                   music_on, music_off, blind_up, blind_down
                   Reply as JSON: {action, entity, params}"
        response_variable: ki_response
      - choose:
          - conditions: "{{ 'light' in ki_response.text }}"
            sequence:
              - service: light.toggle
                data:
                  entity_id: "{{ ki_response.entity }}"

Practical example 2: Intelligent temperature control

automation:
  - alias: "AI climate"
    trigger:
      - platform: time_pattern
        minutes: "/15"
    action:
      - service: ollama.generate
        data:
          prompt: |
            Current data:
            - Outside temperature: {{ states('sensor.outside_temperature') }}°C
            - Inside temperature: {{ states('sensor.inside_temperature') }}°C
            - Humidity: {{ states('sensor.humidity') }}%
            - Present: {{ states('person.max') }}
            - Time: {{ now().strftime('%H:%M') }}
            - Day: {{ now().strftime('%A') }}

            What should heating/cooling do?
            Answer: heat, cool, off, or ventilate
            + brief reason
        response_variable: ki_decision
      - choose:
          - conditions: "{{ 'heat' in ki_decision.text }}"
            sequence:
              - service: climate.set_hvac_mode
                data:
                  hvac_mode: heat
          - conditions: "{{ 'cool' in ki_decision.text }}"
            sequence:
              - service: climate.set_hvac_mode
                data:
                  hvac_mode: cool

Practical example 3: Morning briefing

automation:
  - alias: "Morning briefing"
    trigger:
      - platform: time
        at: "07:30:00"
    action:
      - service: ollama.generate
        data:
          prompt: |
            Create a short morning briefing:
            - Weather: {{ states('weather.home') }}
            - Outside temperature: {{ states('sensor.outside_temperature') }}°C
            - Today: {{ now().strftime('%A, %d.%m.%Y') }}
            - Calendar: {{ states('calendar.personal') }}
            - Electricity price: {{ states('sensor.electricity_price') }} cents/kWh

            Summarize everything in 3 sentences.
        response_variable: briefing
      - service: notify.notify
        data:
          title: "Morning briefing"
          message: "{{ briefing.text }}"
      - service: tts.speak
        data:
          message: "{{ briefing.text }}"
          media_player_entity_id: media_player.kitchen

Practical Example 4: Security Analysis

automation:
  - alias: "AI Security Analysis"
    trigger:
      - platform: state
        entity_id: binary_sensor.motion
        to: "on"
    condition:
      - condition: time
        after: "23:00"
        before: "06:00"
    action:
      - service: ollama.generate
        data:
          prompt: |
            Motion detected at night:
            - Sensor: {{ trigger.entity_id }}
            - Time: {{ now().strftime('%H:%M') }}
            - People home: {{ states('group.people') }}
            - Recent alerts: {{ states('sensor.recent_alerts') }}

            Is this suspicious? Reply: normal/suspicious + reason
        response_variable: analysis
      - choose:
          - conditions: "{{ 'suspicious' in analysis.text }}"
            sequence:
              - service: notify.notify
                data:
                  title: "Security Alert"
                  message: "{{ analysis.text }}"
              - service: camera.snapshot
                data:
                  entity_id: camera.hallway

Security Considerations

• Everything local: Ollama, Whisper, Piper, all running locally. No cloud required.
• Tokens: Keep Home Assistant tokens for Ollama integration to a minimum.
• Validation: AI decisions for critical actions (unlocking doors, triggering alarms) should require human approval. See Human Authorization.
• Logging: Record AI decisions for audit trails. See Logging.

Common Pitfalls

• Model too slow: Large models (>13B) are too sluggish for real-time speech. Use llama3.1:8b or phi3 instead.
• Whisper model selection: small strikes a good balance. large is more accurate but slower.
• German voice: For Piper, de_DE-thorsten-high delivers good quality.
• Prompt too long: Too many entities in a single prompt will overwhelm the model. Filter to what’s relevant.
• AI misinterpretation: “Turn on light” can affect multiple entities. Use precise prompts with explicit entity lists.

Further Reading

• Node-RED Home Assistant - Node-RED and Home Assistant.
• Home Automation - Smart homes with AI.
• Ollama - Model server.
• Whisper - Speech-to-Text.
• Voice Assistants - Local voice AI.
• Ollama Integrations - Integration with other tools.

Key Takeaways:

• Home Assistant + Ollama equals a fully local voice assistant.
• Whisper for STT, Ollama for understanding, Piper for TTS.
• Use AI in automations to make intelligent decisions.
• Everything stays local, no cloud, no data leaves your network.
• Smaller models (8B) for real-time, larger ones for analysis.

FAQ

References and Further Reading

• Home Assistant Ollama Integration - Official integration.
• Home Assistant Assist - Voice assistant.
• Wyoming - Speech services protocol.
• Ollama - Local model server.

Premium VPS Hosting

Schnelle und sichere VPS Server

Gesponsert