5. TTS: ElevenLabs and Others

Description

ElevenLabs is the premium TTS provider: ultra-realistic voices, emotional control, voice cloning, and a catalog of community voices. There are also Google Cloud TTS and Amazon Polly as cloud alternatives. In this capsule you'll compare all the TTS providers, implement working code for each one, and build a unified system with fallback that lets you switch providers with a single parameter.

Why it matters: OpenAI TTS is good, but it isn't always the best option. ElevenLabs produces more expressive and natural audio for published content. Google TTS supports more languages and dialects. Amazon Polly is cheaper for high volume. Knowing the alternatives lets you choose the right provider based on quality, cost and the required features.

Connection with the project: The Audio Pipeline (capsule 08) uses OpenAI TTS by default, but you can extend it to use ElevenLabs when premium quality is needed, or Google TTS for languages OpenAI doesn't handle well.

ElevenLabs

Setup

pip install elevenlabs

export ELEVENLABS_API_KEY="your_key"

Current API (SDK v1+)

from elevenlabs.client import ElevenLabs

el_client = ElevenLabs(api_key=os.getenv("ELEVENLABS_API_KEY"))

def tts_elevenlabs(
    text: str,
    voice_id: str = "21m00Tcm4TlvDq8ikWAM",
    model_id: str = "eleven_multilingual_v2",
    output_path: str = "elevenlabs_output.mp3"
) -> str:
    audio = el_client.generate(
        text=text,
        voice=voice_id,
        model=model_id
    )

    with open(output_path, "wb") as f:
        for chunk in audio:
            f.write(chunk)

    return output_path

Popular ElevenLabs voices

List available voices

def list_elevenlabs_voices() -> list[dict]:
    voices = el_client.voices.get_all()
    return [
        {
            "voice_id": voice.voice_id,
            "name": voice.name,
            "category": voice.category,
            "labels": voice.labels
        }
        for voice in voices.voices
    ]

ElevenLabs models

def tts_elevenlabs_turbo(text: str, voice_id: str, output_path: str) -> str:
    audio = el_client.generate(
        text=text,
        voice=voice_id,
        model="eleven_turbo_v2_5"
    )
    with open(output_path, "wb") as f:
        for chunk in audio:
            f.write(chunk)
    return output_path

Voice Settings (style control)

from elevenlabs import VoiceSettings

def tts_elevenlabs_custom(
    text: str,
    voice_id: str,
    stability: float = 0.5,
    similarity_boost: float = 0.75,
    style: float = 0.0,
    output_path: str = "custom_output.mp3"
) -> str:
    audio = el_client.generate(
        text=text,
        voice=voice_id,
        voice_settings=VoiceSettings(
            stability=stability,
            similarity_boost=similarity_boost,
            style=style,
            use_speaker_boost=True
        ),
        model="eleven_multilingual_v2"
    )
    with open(output_path, "wb") as f:
        for chunk in audio:
            f.write(chunk)
    return output_path

Voice Settings parameters:

Voice Design (create a custom voice)

def design_voice(
    gender: str = "female",
    age: str = "young",
    accent: str = "american",
    accent_strength: float = 1.0,
    text: str = "Hello, this is a test of voice design."
) -> bytes:
    audio = el_client.voice_generation.generate_voice(
        gender=gender,
        age=age,
        accent=accent,
        accent_strength=accent_strength,
        text=text
    )
    return audio

Google Cloud Text-to-Speech

Setup

pip install google-cloud-texttospeech

Basic synthesis

from google.cloud import texttospeech

def tts_google(
    text: str,
    language: str = "es-ES",
    voice_name: str = "es-ES-Neural2-A",
    output_path: str = "google_tts.mp3"
) -> str:
    client = texttospeech.TextToSpeechClient()

    input_text = texttospeech.SynthesisInput(text=text)
    voice = texttospeech.VoiceSelectionParams(
        language_code=language,
        name=voice_name,
    )
    audio_config = texttospeech.AudioConfig(
        audio_encoding=texttospeech.AudioEncoding.MP3,
        speaking_rate=1.0,
        pitch=0.0,
    )

    response = client.synthesize_speech(
        input=input_text,
        voice=voice,
        audio_config=audio_config,
    )

    with open(output_path, "wb") as f:
        f.write(response.audio_content)

    return output_path

Google TTS voices for Spanish

List available voices

def list_google_voices(language: str = "es") -> list[dict]:
    client = texttospeech.TextToSpeechClient()
    voices = client.list_voices(language_code=language)

    return [
        {
            "name": voice.name,
            "language": voice.language_codes[0],
            "gender": texttospeech.SsmlVoiceGender(voice.ssml_gender).name,
            "sample_rate": voice.natural_sample_rate_hertz,
        }
        for voice in voices.voices
    ]

Google TTS with SSML

Google TTS supports SSML (Speech Synthesis Markup Language) for advanced control:

def tts_google_ssml(ssml: str, output_path: str = "google_ssml.mp3") -> str:
    client = texttospeech.TextToSpeechClient()

    input_text = texttospeech.SynthesisInput(ssml=ssml)
    voice = texttospeech.VoiceSelectionParams(
        language_code="es-ES",
        name="es-ES-Neural2-A",
    )
    audio_config = texttospeech.AudioConfig(
        audio_encoding=texttospeech.AudioEncoding.MP3
    )

    response = client.synthesize_speech(
        input=input_text,
        voice=voice,
        audio_config=audio_config,
    )

    with open(output_path, "wb") as f:
        f.write(response.audio_content)

    return output_path

ssml_example = """
<speak>
  Welcome to the report.
  <break time="500ms"/>
  <emphasis level="strong">Key point number one:</emphasis>
  Sales grew by <say-as interpret-as="cardinal">25</say-as> percent.
  <break time="300ms"/>
  <prosody rate="slow">This is very important.</prosody>
</speak>
"""

Amazon Polly

Setup

pip install boto3

Basic generation

import boto3

def tts_polly(
    text: str,
    voice_id: str = "Lucia",
    engine: str = "neural",
    output_path: str = "polly_output.mp3"
) -> str:
    polly = boto3.client("polly")

    response = polly.synthesize_speech(
        Text=text,
        OutputFormat="mp3",
        VoiceId=voice_id,
        Engine=engine,
    )

    with open(output_path, "wb") as f:
        f.write(response["AudioStream"].read())

    return output_path

Polly voices for Spanish

Complete Comparison of TTS Providers

Comparison table

Comparative costs

TTS Decision Tree

Do you need TTS?
│
├── Is the priority maximum quality and expressiveness?
│   ├── Yes → ElevenLabs (eleven_multilingual_v2)
│   └── No → continue ↓
│
├── Do you need voice cloning?
│   ├── Yes → ElevenLabs (the only one with voice cloning)
│   └── No → continue ↓
│
├── Do you already use the OpenAI ecosystem?
│   ├── Yes → Do you need HD quality?
│   │        ├── Yes → OpenAI tts-1-hd
│   │        └── No → OpenAI tts-1
│   └── No → continue ↓
│
├── Do you need SSML for fine-grained control?
│   ├── Yes → Google TTS or Amazon Polly
│   └── No → continue ↓
│
├── Do you need maximum volume at the lowest cost?
│   ├── Yes → Amazon Polly Standard ($4/1M chars)
│   └── No → continue ↓
│
├── Do you need the largest free tier?
│   ├── Yes → Amazon Polly (5M chars free) or Google TTS (1M chars free)
│   └── No → continue ↓
│
└── Default → OpenAI tts-1 (simple, integrated, good quality)

Unified Multi-Provider TTS Function

from enum import Enum

class TTSProvider(Enum):
    OPENAI = "openai"
    ELEVENLABS = "elevenlabs"
    GOOGLE = "google"
    POLLY = "polly"

def tts_unified(
    text: str,
    provider: TTSProvider = TTSProvider.OPENAI,
    voice: str = None,
    output_path: str = "output.mp3"
) -> dict:
    default_voices = {
        TTSProvider.OPENAI: "nova",
        TTSProvider.ELEVENLABS: "21m00Tcm4TlvDq8ikWAM",
        TTSProvider.GOOGLE: "es-ES-Neural2-A",
        TTSProvider.POLLY: "Lucia",
    }

    voice = voice or default_voices[provider]

    if provider == TTSProvider.OPENAI:
        response = client.audio.speech.create(
            model="tts-1", voice=voice, input=text
        )
        response.stream_to_file(output_path)

    elif provider == TTSProvider.ELEVENLABS:
        audio = el_client.generate(
            text=text, voice=voice, model="eleven_multilingual_v2"
        )
        with open(output_path, "wb") as f:
            for chunk in audio:
                f.write(chunk)

    elif provider == TTSProvider.GOOGLE:
        tts_google(text, voice_name=voice, output_path=output_path)

    elif provider == TTSProvider.POLLY:
        tts_polly(text, voice_id=voice, output_path=output_path)

    return {
        "output_path": output_path,
        "provider": provider.value,
        "voice": voice,
        "characters": len(text)
    }

TTS with Fallback

def tts_with_fallback(
    text: str,
    providers: list[TTSProvider] = None,
    output_path: str = "output.mp3"
) -> dict:
    if providers is None:
        providers = [TTSProvider.OPENAI, TTSProvider.ELEVENLABS, TTSProvider.GOOGLE]

    errors = []
    for provider in providers:
        try:
            result = tts_unified(text, provider=provider, output_path=output_path)
            result["fallback_used"] = provider != providers[0]
            result["attempts"] = len(errors) + 1
            return result
        except Exception as e:
            errors.append({"provider": provider.value, "error": str(e)})

    return {"error": "All providers failed", "details": errors}

Troubleshooting

Problem 1: ElevenLabs — voice not found

Symptom: Voice not found or Invalid voice_id.

Solution:

voices = list_elevenlabs_voices()
valid_ids = [v["voice_id"] for v in voices]
print(f"Available voices: {len(valid_ids)}")

Problem 2: Google TTS — credentials not configured

Symptom: DefaultCredentialsError.

Solution: Verify that GOOGLE_APPLICATION_CREDENTIALS points to a valid service-account JSON file.

Problem 3: ElevenLabs — plan character limit

Symptom: Quota exceeded or error 429.

Solution: Check the current plan and remaining characters:

def check_elevenlabs_usage() -> dict:
    user = el_client.user.get()
    sub = user.subscription
    return {
        "tier": sub.tier,
        "character_count": sub.character_count,
        "character_limit": sub.character_limit,
        "remaining": sub.character_limit - sub.character_count
    }

Problem 4: Generated audio sounds robotic

Symptom: Voice without naturalness, mechanical pronunciation.

Exercises

Exercise 1: A/B provider comparator

Create a function that generates the same text with OpenAI and ElevenLabs, measures latency, and returns the paths for auditory comparison.

import time

def compare_tts_providers(
    text: str,
    output_dir: str = "tts_comparison"
) -> list[dict]:
    from pathlib import Path
    Path(output_dir).mkdir(exist_ok=True)

    providers_config = [
        {
            "provider": TTSProvider.OPENAI,
            "voice": "nova",
            "path": f"{output_dir}/openai_nova.mp3"
        },
        {
            "provider": TTSProvider.ELEVENLABS,
            "voice": "21m00Tcm4TlvDq8ikWAM",
            "path": f"{output_dir}/elevenlabs_rachel.mp3"
        },
    ]

    results = []
    for config in providers_config:
        start = time.time()
        try:
            tts_unified(
                text,
                provider=config["provider"],
                voice=config["voice"],
                output_path=config["path"]
            )
            elapsed = time.time() - start
            file_size = Path(config["path"]).stat().st_size
            results.append({
                "provider": config["provider"].value,
                "voice": config["voice"],
                "path": config["path"],
                "latency_s": round(elapsed, 2),
                "file_size_kb": round(file_size / 1024, 1),
                "status": "success"
            })
        except Exception as e:
            results.append({
                "provider": config["provider"].value,
                "status": "error",
                "error": str(e)
            })

    return results

Exercise 2: Multi-voice audiobook generator

Create a function that takes text with character markers ([NARRATOR], [ANA], [PEDRO]) and generates audio with a different voice for each character.

import re

def generate_multivoice_audiobook(
    script: str,
    voice_map: dict[str, dict] = None,
    output_path: str = "audiobook.mp3"
) -> dict:
    if voice_map is None:
        voice_map = {
            "NARRATOR": {"provider": TTSProvider.OPENAI, "voice": "echo"},
            "ANA": {"provider": TTSProvider.OPENAI, "voice": "nova"},
            "PEDRO": {"provider": TTSProvider.OPENAI, "voice": "onyx"},
        }

    pattern = r'\[([A-ZÁÉÍÓÚÑ]+)\]\s*'
    parts = re.split(pattern, script)

    audio_segments = []
    current_speaker = "NARRATOR"

    with tempfile.TemporaryDirectory() as tmp_dir:
        seg_idx = 0
        i = 0
        while i < len(parts):
            part = parts[i].strip()
            if part in voice_map:
                current_speaker = part
                i += 1
                continue

            if not part:
                i += 1
                continue

            config = voice_map.get(current_speaker, voice_map["NARRATOR"])
            seg_path = str(Path(tmp_dir) / f"seg_{seg_idx:03d}.mp3")

            tts_unified(
                part,
                provider=config["provider"],
                voice=config["voice"],
                output_path=seg_path
            )
            audio_segments.append(AudioSegment.from_mp3(seg_path))
            audio_segments.append(AudioSegment.silent(duration=400))
            seg_idx += 1
            i += 1

        if not audio_segments:
            return {"error": "No audio segments were generated"}

        combined = audio_segments[0]
        for seg in audio_segments[1:]:
            combined += seg

        combined.export(output_path, format="mp3")

    return {
        "output_path": output_path,
        "segments": seg_idx,
        "speakers": list(set(voice_map.keys())),
        "duration_seconds": len(combined) / 1000
    }

Exercise 3: Multi-provider cost calculator

Create a function that, given a text and a list of providers, calculates the estimated cost with each one and recommends the most economical.

def calculate_tts_costs(text: str) -> dict:
    char_count = len(text)

    costs = {
        "openai_tts1": {
            "provider": "OpenAI tts-1",
            "cost_per_1m": 15.00,
            "estimated_usd": round(char_count * 0.000015, 6)
        },
        "openai_tts1hd": {
            "provider": "OpenAI tts-1-hd",
            "cost_per_1m": 30.00,
            "estimated_usd": round(char_count * 0.000030, 6)
        },
        "elevenlabs_starter": {
            "provider": "ElevenLabs (Starter)",
            "cost_per_1m": 22.00,
            "estimated_usd": round(char_count * 0.000022, 6)
        },
        "google_neural": {
            "provider": "Google Neural",
            "cost_per_1m": 16.00,
            "estimated_usd": round(char_count * 0.000016, 6)
        },
        "polly_neural": {
            "provider": "Amazon Polly Neural",
            "cost_per_1m": 16.00,
            "estimated_usd": round(char_count * 0.000016, 6)
        },
        "polly_standard": {
            "provider": "Amazon Polly Standard",
            "cost_per_1m": 4.00,
            "estimated_usd": round(char_count * 0.000004, 6)
        },
    }

    cheapest = min(costs.values(), key=lambda x: x["estimated_usd"])

    return {
        "characters": char_count,
        "costs": costs,
        "cheapest": cheapest["provider"],
        "cheapest_cost": cheapest["estimated_usd"]
    }

Summary

• ElevenLabs is the leader in TTS quality: ultra-realistic voices, voice cloning, emotional control, 29 languages.
• Google Cloud TTS offers the widest variety of voices (200+) and full SSML support.
• Amazon Polly is the most economical option for high volume ($4/1M chars in Standard).
• OpenAI TTS is the simplest option if you already use the OpenAI ecosystem.
• Use the unified function tts_unified() to switch providers with a single parameter.
• Implement multi-provider fallback for resilience in production.
• For Spanish, ElevenLabs with eleven_multilingual_v2 offers the best naturalness.

Additional Resources

1. ElevenLabs Docs — Official documentation
2. ElevenLabs Python SDK — Official SDK
3. Google Cloud TTS — Google's documentation
4. Amazon Polly — AWS documentation
5. SSML Reference — SSML reference