> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/yocxy2/chatterboxyocxy/llms.txt
> Use this file to discover all available pages before exploring further.

# Voice Conversion

> Convert any voice to match a target speaker using ChatterboxVC

## Overview

ChatterboxVC provides voice conversion capabilities, allowing you to transform the voice characteristics of any audio file to match a target speaker while preserving the original speech content and prosody. Unlike text-to-speech, voice conversion works directly with audio input.

<CardGroup cols={2}>
  <Card title="Voice Transformation" icon="wand-magic-sparkles">
    Convert any speaker's voice to match your target voice while keeping the original content.
  </Card>

  <Card title="Prosody Preservation" icon="wave-square">
    Maintains the original timing, rhythm, and intonation of the source audio.
  </Card>

  <Card title="Zero-Shot" icon="bolt">
    No training required - just provide a target voice reference.
  </Card>

  <Card title="High Quality" icon="star">
    24kHz output with natural voice transformation.
  </Card>
</CardGroup>

## Voice Conversion vs TTS

Understand the key differences between voice conversion and text-to-speech:

| Aspect       | Voice Conversion (VC) | Text-to-Speech (TTS)  |
| ------------ | --------------------- | --------------------- |
| **Input**    | Audio file            | Text string           |
| **Output**   | Transformed audio     | Generated speech      |
| **Content**  | Preserves original    | Creates new content   |
| **Prosody**  | Keeps original timing | Generates new prosody |
| **Use Case** | Voice transformation  | Speech synthesis      |

<Note>
  Voice conversion is ideal when you want to change who is speaking while keeping the exact timing, emotion, and delivery of the original performance.
</Note>

## Model Specifications

* **Input**: Audio file (automatically resampled to 16kHz)
* **Output Sample Rate**: 24,000 Hz
* **Architecture**: S3Gen decoder with voice conditioning
* **Repository**: `ResembleAI/chatterbox`

## Hardware Requirements

<CardGroup cols={2}>
  <Card title="Minimum (CPU)" icon="microchip">
    * 4GB RAM
    * CPU inference supported
    * Slower conversion times
  </Card>

  <Card title="Recommended (GPU)" icon="server">
    * NVIDIA GPU with 4GB+ VRAM
    * CUDA support
    * Real-time conversion possible
  </Card>
</CardGroup>

<Note>
  The model also supports Apple Silicon (MPS) for Mac users with M1/M2/M3 chips.
</Note>

## Usage

### Basic Voice Conversion

```python theme={null}
import torch
import torchaudio as ta
from chatterbox.vc import ChatterboxVC

# Load the voice conversion model
model = ChatterboxVC.from_pretrained(device="cuda")

# Convert voice
source_audio = "original_speaker.wav"
target_voice = "desired_voice.wav"

wav = model.generate(
    audio=source_audio,
    target_voice_path=target_voice
)

ta.save("converted.wav", wav, model.sr)
```

### Auto-detect Device

```python theme={null}
import torch
import torchaudio as ta
from chatterbox.vc import ChatterboxVC

# Automatically detect the best available device
if torch.cuda.is_available():
    device = "cuda"
elif torch.backends.mps.is_available():
    device = "mps"
else:
    device = "cpu"

print(f"Using device: {device}")

model = ChatterboxVC.from_pretrained(device=device)

source_audio = "source.wav"
target_voice = "target.wav"

wav = model.generate(
    audio=source_audio,
    target_voice_path=target_voice
)

ta.save("converted.wav", wav, model.sr)
```

### Batch Processing

```python theme={null}
import torch
import torchaudio as ta
from chatterbox.vc import ChatterboxVC
from pathlib import Path

model = ChatterboxVC.from_pretrained(device="cuda")

# Set target voice once
target_voice = "celebrity_voice.wav"
model.set_target_voice(target_voice)

# Process multiple files with the same target voice
source_files = [
    "speaker1.wav",
    "speaker2.wav",
    "speaker3.wav"
]

for i, source_file in enumerate(source_files):
    wav = model.generate(audio=source_file)
    ta.save(f"converted_{i}.wav", wav, model.sr)
```

<Tip>
  When processing multiple files with the same target voice, use `set_target_voice()` once and then call `generate()` without the `target_voice_path` parameter for better performance.
</Tip>

### Pre-setting Target Voice

```python theme={null}
import torchaudio as ta
from chatterbox.vc import ChatterboxVC

model = ChatterboxVC.from_pretrained(device="cuda")

# Pre-set the target voice
model.set_target_voice("my_target_voice.wav")

# Generate without specifying target_voice_path each time
wav = model.generate(audio="source1.wav")
ta.save("converted1.wav", wav, model.sr)

wav = model.generate(audio="source2.wav")
ta.save("converted2.wav", wav, model.sr)
```

## How It Works

<Steps>
  <Step title="Audio Tokenization">
    The source audio is converted to 16kHz and tokenized using the S3 tokenizer, which extracts semantic speech features.
  </Step>

  <Step title="Voice Embedding">
    The target voice reference (first 10 seconds) is embedded to capture the speaker's voice characteristics.
  </Step>

  <Step title="Voice Transformation">
    The S3Gen decoder transforms the source audio tokens using the target voice embedding while preserving the original content and prosody.
  </Step>

  <Step title="Audio Synthesis">
    The transformed tokens are decoded to 24kHz audio with the target voice characteristics.
  </Step>
</Steps>

## Generation Parameters

| Parameter           | Type        | Description                                          |
| ------------------- | ----------- | ---------------------------------------------------- |
| `audio`             | str         | Path to source audio file to convert                 |
| `target_voice_path` | str \| None | Path to target voice reference (optional if pre-set) |

<Note>
  Unlike TTS models, voice conversion has minimal parameters since it preserves the prosody and timing of the original audio.
</Note>

## Best Practices

### Reference Audio Quality

<CardGroup cols={2}>
  <Card title="Target Voice" icon="microphone">
    * Use clean, noise-free audio
    * 5-10 seconds of speech
    * Clear, natural speaking
    * Representative of desired voice
  </Card>

  <Card title="Source Audio" icon="file-audio">
    * Any length supported
    * Automatically resampled
    * Speech-only recommended
    * Minimize background noise
  </Card>
</CardGroup>

### Optimal Results

1. **Clean Audio**: Both source and target should be free of background noise
2. **Similar Speaking Styles**: Better results when source and target have similar speaking rates
3. **Quality References**: Use high-quality recordings for the target voice
4. **Speech Content**: Works best with speech-only audio (no music or sound effects)

<Warning>
  Voice conversion quality depends heavily on the quality of both the source audio and target voice reference. Poor quality inputs will result in degraded outputs.
</Warning>

## Technical Details

### Audio Processing Pipeline

```python theme={null}
# Internal processing flow (for reference)
# 1. Load source audio and resample to 16kHz
audio_16k, _ = librosa.load(source_audio, sr=16000)

# 2. Tokenize source audio
speech_tokens, _ = tokenizer(audio_16k)

# 3. Load target voice and extract embeddings
target_audio, _ = librosa.load(target_voice, sr=24000)
ref_dict = s3gen.embed_ref(target_audio[:240000])  # First 10 seconds

# 4. Generate with voice transformation
output_wav = s3gen.inference(speech_tokens, ref_dict)
```

### Conditioning Length

* **Source Audio**: Full length is processed (no limit)
* **Target Voice**: First 10 seconds (240,000 samples at 24kHz) are used for voice conditioning

## Built-in Watermarking

Every audio file generated by ChatterboxVC includes Resemble AI's Perth (Perceptual Threshold) watermark - imperceptible neural watermarks that survive MP3 compression, audio editing, and common manipulations.

```python theme={null}
import perth
import librosa

# Load the converted audio
converted_audio, sr = librosa.load("converted.wav", sr=None)

# Initialize watermarker
watermarker = perth.PerthImplicitWatermarker()

# Extract watermark
watermark = watermarker.get_watermark(converted_audio, sample_rate=sr)
print(f"Extracted watermark: {watermark}")  # 0.0 or 1.0
```

## Use Cases

* **Content Localization**: Adapt voice actors for different regions while keeping performances
* **Voice Replacement**: Replace placeholder voices in video production
* **Privacy Protection**: Anonymize speakers while preserving speech content
* **Character Consistency**: Maintain consistent character voices across recordings
* **Audio Restoration**: Update old recordings with clearer voices
* **Voice Acting**: Transform voice performances to match different characters

## Performance Characteristics

<CardGroup cols={2}>
  <Card title="Conversion Speed" icon="clock">
    Fast processing with efficient tokenization. Real-time or near-real-time on modern GPUs.
  </Card>

  <Card title="Audio Quality" icon="star">
    High-fidelity 24kHz output that preserves original prosody while transforming voice characteristics.
  </Card>
</CardGroup>

## Limitations

<Warning>
  * Voice conversion works best with clean, speech-only audio
  * Background music or noise may affect quality
  * Extreme voice transformations (e.g., male to female or vice versa) may sound less natural
  * The model cannot add emotions or change prosody - it only transforms voice timbre
</Warning>

## Comparison with TTS

When should you use voice conversion vs TTS?

<CardGroup cols={2}>
  <Card title="Use Voice Conversion When:" icon="check">
    * You have audio you want to transform
    * You need to preserve exact timing
    * You want to keep original performance
    * You're replacing voices in existing content
  </Card>

  <Card title="Use TTS When:" icon="check">
    * You're starting from text
    * You need to generate new speech
    * You want to control prosody
    * You're creating original content
  </Card>
</CardGroup>

## Next Steps

<CardGroup cols={2}>
  <Card title="Installation" icon="download" href="/quickstart">
    Install Chatterbox and get started
  </Card>

  <Card title="TTS Models" icon="microphone" href="/models/chatterbox-turbo">
    Explore text-to-speech models
  </Card>
</CardGroup>