OpenVoice: Clone Voices with Local Open-Source AI

What This Article Covers

• What OpenVoice from MyShell can do and how the two-stage technique works.
• Installation, demo, and generating your first voice outputs.
• Style control: changing emotion, accent, and speech rhythm.
• The limitation for German speakers and the XTTS alternative.
• Legal and ethical considerations: why voice cloning requires consent.

What OpenVoice Is

OpenVoice is an open-source project from MyShell that brings voice cloning to your own hardware. The core is a two-stage technique: a standard text-to-speech engine first generates audio in a neutral base voice, and a second model called the Tone Color Converter applies the vocal characteristics of a reference person on top. The result sounds as though the reference person spoke the text, and the model needs only a short audio sample from them.

The key advantage over other cloning tools is style control: OpenVoice separates tone color from everything else. You can make the same cloned voice sound happy, sad, whispered, or rhythmically different without losing the voice itself. The project is MIT-licensed, runs entirely locally, and represents one of the few free alternatives to cloud services like ElevenLabs.

V1 and V2: The Differences

• OpenVoice V1 arrived in early 2024 and supported only English and Chinese. The Tone Color Converter was the technical innovation: clone vocal characteristics without retraining the base model.
• OpenVoice V2 significantly improved audio quality and brought multilingual support via MeloTTS, also from MyShell. V2 now provides base models for English, Spanish, French, Chinese, Japanese, and Korean.

The Limitation for German Speakers

Here’s the critical point for German audiences: OpenVoice V2 does not support German as a base language. MeloTTS, which provides the base voices, has no German model. In practical terms: you can clone a voice, but the text must be output in one of the six supported languages, otherwise it will sound off.

For German-language voice cloning, the more practical local option is currently Coqui XTTS v2, which supports German among seventeen languages and also clones voices using a reference sample. If you want to clone German voices, use XTTS. If you need English or one of the six V2 languages, OpenVoice is the leaner tool. For a general overview of local speech synthesis, see the article on text-to-speech models.

Installation

OpenVoice runs best on Linux or WSL with an NVIDIA GPU, but also works on CPU, just significantly slower:

git clone https://github.com/myshell-ai/OpenVoice.git
cd OpenVoice
conda create -n openvoice python=3.9
conda activate openvoice
pip install -e .

Then download the checkpoints: grab V1 or V2 from HuggingFace and extract them into the checkpoints or checkpoints_v2 directory respectively. For V2, also run pip install melotts for multilingual support. The installation pulls several GB of model files, so allow time for it to complete.

Your First Voice Output

The quickest approach is the included Gradio demo:

python openvoice_app.py --share

This launches a web interface where you enter text, upload a reference audio file, and adjust parameters. For your own integration, the Python code looks like this:

from openvoice import se_extractor
from openvoice.api import BaseSpeakerTTS, ToneColorConverter

# 1. Extract tone color from reference audio
target_se, _ = se_extractor.get_se(
    "referenz.wav", tone_color_converter, vad=True
)

# 2. Convert text to base voice
base_speaker_tts.tts(
    "This is a test.", "tmp.wav", speaker="default", language="English"
)

# 3. Apply reference voice on top
tone_color_converter.convert(
    audio_src_path="tmp.wav",
    src_se=src_se,
    tgt_se=target_se,
    output_path="cloned.wav"
)

The reference audio should be ten to thirty seconds of clean speech without music or background noise. The clearer the source, the more precise the cloning.

Style Control

What sets OpenVoice apart from most cloning tools: you can change emotional tone and rhythm without losing the voice. Parameters like speed and style controls in V1 for English let you adjust intonation, speaking pace, and pauses. V2 streamlines style control in favor of better audio quality and more languages. In practice, this means you clone once, then use that same voice for different tones, from calm explanation to energetic pitch.

Practical Use Cases

• Video voiceover: use your own voice for YouTube content without recording every time. This fits well with faceless YouTube automation.
• Audiobook and podcast: generate long texts in your own voice.
• Voice assistant: use your voice for a local assistant, combined with local voice assistants.
• Prototyping: test voices for characters or demos before hiring a voice actor.
• Multilingual output: use a single reference voice for outputs in English, Spanish, and the other V2 languages.

Legal and Ethical Considerations

Most importantly: cloning someone else’s voice without consent is legally problematic and ethically wrong. In Germany, the voice falls under personality rights, and cloned voices used for fraud, misinformation, or defamation are punishable. Even for private experiments, only use your own voice or get explicit permission. If you use OpenVoice in production, label outputs as AI-generated; platforms like YouTube increasingly require this anyway.

Frequently Asked Questions

What can OpenVoice do?

OpenVoice clones a person’s tone color from a short audio sample and applies it to text-to-speech outputs. It can also control style, emotion, and speaking pace without losing the cloned voice.

Does OpenVoice support German?

No, and this is the key limitation. V2 supports English, Spanish, French, Chinese, Japanese, and Korean. For German, Coqui XTTS v2 is the better local choice.

Is OpenVoice free?

Yes, the project is MIT-licensed and runs entirely locally. Costs arise only from hardware, and a GPU significantly speeds up the process.

How long should the reference audio be?

Ten to thirty seconds of clean speech usually suffice. What matters is that the recording is free of music, echo, and background noise, otherwise these flaws carry into the clone.

Can OpenVoice use my voice for YouTube?

Yes, and this is one of the best use cases: clone your voice once, then convert scripts to audio without recording each time. Just remember to label AI-generated content as required.

Do I need a GPU?

Not strictly necessary, but strongly recommended. Conversion runs on CPU but much more slowly. For regular use or real-time applications, an NVIDIA GPU is nearly essential.

How does style control work?

OpenVoice separates tone color from style parameters like emotion, accent, tempo, and pacing. The same cloned voice can sound happy, calm, or emphatic without losing its identity.

How does it compare to ElevenLabs?

ElevenLabs is a cloud service with better audio quality and more languages, including German. OpenVoice is free, local, and open-source, but limited to six languages. Choose OpenVoice for privacy, ElevenLabs for quality and German support.

Is voice cloning legal?

Your own voice, yes. Others’ voices only with consent. Voice is protected as a personality right, and misuse for fraud or false statements is punishable. In production, label the AI origin.

How can you tell a cloned voice?

Quality depends heavily on the reference audio. In V1 and V2, careful listening usually reveals machine origins. For podcasts and videos it often suffices, but cloud services remain ahead for professional use.

Can I use the voice for my assistant?

Yes, if the pipeline is right: OpenVoice for output, a local STT for input, an LLM for answers. Your assistant speaks with your own voice, locally, without the cloud.

What is MeloTTS?

MeloTTS is MyShell’s text-to-speech model that provides V2’s base voices. It covers the six supported languages and is why V2 became multilingual, though German is not included.

How fast is the conversion?

On a decent GPU, it takes seconds per section. On CPU it can take minutes depending on length. For long texts, split them into paragraphs and batch them.

Are there alternatives for German?

Yes: Coqui XTTS v2 for German voice cloning, Piper and eSpeak for fast local TTS without cloning, and cloud alternatives like ElevenLabs or Cartesia for best quality. OpenVoice remains the strongest free option for its supported languages.

How often is OpenVoice updated?

The project made two major jumps with V1 and V2 and remains active, but updates follow no fixed schedule. Check GitHub releases and issues to see the current state.

When OpenVoice Makes Sense

OpenVoice is the right tool if you want local, free voice cloning and the six supported languages work for you. Style control makes it the strongest open-source offering in its class, but the lack of German base models is a boundary that affects many readers. For German cloning, switch to XTTS. For English or the other languages, OpenVoice gives you an MIT-licensed tool that runs on your own hardware and keeps your voice where it belongs: under your control.