Build an AI Voicemail Generator Fast with the ElevenLabs API

Product demos, marketing campaigns, or automated customer service often need a professional, natural-sounding voice message. Recording manually is time-consuming and makes it hard to keep multi-language versions consistent. With ElevenLabs' text-to-speech (TTS) API, you can build a voicemail generator in minutes: input a text template and dynamic variables (such as name and appointment time), and get an MP3 file ready to upload to your phone system.

Based on a public technical tutorial, this article provides a complete path from getting an API key to Python/Node.js implementations, and finally to deploying as a micro REST API. The code examples have been logically reviewed and can serve directly as a project starting point.

Why Choose ElevenLabs for Building a Voice Generator

ElevenLabs offers near-human speech synthesis and voice cloning, and its free tier is enough for prototype validation. Its TTS endpoint accepts plain text and automatically handles punctuation pauses and intonation, so developers don't need to call additional speech synthesis engines.

The core workflow has only three steps: fill the template → call ElevenLabs → save the MP3. This model can be easily embedded in webhooks, CI pipelines, or CLI tools.

Technical background on text-to-speech (TTS) and voice cloning
ElevenLabs' TTS service is based on deep neural networks. It splits input text into phoneme/subword units, predicts acoustic features, and then synthesizes waveforms with a vocoder. Its "voice cloning" comes in two forms: Instant Voice Cloning and Professional Voice Cloning. The former requires only tens of seconds of samples to quickly generate a similar timbre, while the latter needs longer audio and training time for higher fidelity. Audio returned by the API is encoded as MP3 by default, with sample rate and bitrate determined by the model and request parameters. Developers should distinguish "speech synthesis quality" from "audio post-processing": the former is controlled by the model, stability, similarity_boost, and other parameters, while the latter (fade in/out, loudness normalization) requires ffmpeg/pydub.

Preparation and Getting an API Key

You need four things: an ElevenLabs API key, a Python 3.8+ or Node.js environment, optional ffmpeg (for audio concatenation or post-processing), and a text template.

Getting an API key is straightforward: sign up for an ElevenLabs account, go to API Keys → New Key in the Dashboard, and copy the generated token. It's recommended to store the key in the environment variable ELEVENLABS_API_KEY to avoid hardcoding it in code.

Designing a Reusable Voicemail Template

A typical appointment reminder template looks like this:

Hi {{name}}, this is a reminder from {{company}}.
Your appointment is scheduled for {{date}} at {{time}}.
If you need to reschedule, please call us at {{phone}}.
Thanks, and see you soon!

In Python, you can use str.format to fill placeholders; in JavaScript, template literals work. This design lets the same voice logic be reused across different customers and scenarios.

Python Implementation: From Template to MP3

The Python version relies on requests and pydub, the latter requiring ffmpeg installed on the system. The core synthesize function sends a POST request to https://api.elevenlabs.io/v1/text-to-speech/{voice_id}, with the request body containing the text, model ID, and voice parameters.

import os, requests
ELEVEN_API = os.getenv("ELEVENLABS_API_KEY")
BASE_URL = "https://api.elevenlabs.io/v1"

def synthesize(text, voice_id="EXAVITQu4vr4xnSDxMaL"):
    url = f"{BASE_URL}/text-to-speech/{voice_id}"
    headers = {"xi-api-key": ELEVEN_API, "Content-Type": "application/json"}
    payload = {
        "text": text,
        "model_id": "eleven_monolingual_v1",
        "voice_settings": {"stability": 0.75, "similarity_boost": 0.8}
    }
    response = requests.post(url, json=payload, headers=headers)
    response.raise_for_status()
    return response.content

The default voice_id corresponds to the Rachel voice. If you use a cloned voice, just replace it with the ID returned by the cloning endpoint.

Once you have the audio bytes, load them with pydub's AudioSegment, add post-processing such as fade in/out, and export as MP3. The example adds a 1-second fade in and out to make the beginning and end more natural.

from pydub import AudioSegment
from io import BytesIO

def build_voicemail(data):
    template = ("Hi {name}, this is a reminder from {company}. "
                "Your appointment is scheduled for {date} at {time}. "
                "If you need to reschedule, please call us at {phone}. "
                "Thanks, and see you soon!")
    filled = template.format(**data)
    audio_bytes = synthesize(filled)
    audio = AudioSegment.from_file(BytesIO(audio_bytes), format="mp3")
    audio = audio.fade_in(1000).fade_out(1000)
    output_path = f"voicemail_{data['name'].replace(' ', '_')}.mp3"
    audio.export(output_path, format="mp3")
    print(f"Voicemail saved to {output_path}")

After running the script, you'll get an MP3 file that sounds like a real person reading aloud, ready to use in phone systems or marketing materials.

voice_settings parameters and model selection
The voice_settings in the request body directly affect the listening experience: the lower the stability, the more pronounced the intonation shifts and emotional variation, but it may introduce instability; the higher it is, the smoother and more predictable the output. similarity_boost controls how closely the synthesized speech matches the target timbre; too high may amplify noise in the cloned samples. The example uses eleven_monolingual_v1, a monolingual model for English; for multilingual needs, switch to a multilingual model (such as eleven_multilingual_v2) and ensure the template language matches the languages supported by the model. In addition, voice_id is the unique identifier of a timbre; the default Rachel voice ID is for demonstration only. In production, you should list available voices via the /v1/voices endpoint or use the ID returned by cloning.

Node.js Version: An Equivalent Asynchronous Flow

The Node.js implementation uses node-fetch and fluent-ffmpeg. The synthesize function returns a Buffer of raw MP3 bytes, and buildVoicemail fills the data using template literals and writes to disk.

require('dotenv').config();
const fetch = require('node-fetch');
const fs = require('fs');
const ELEVEN_API = process.env.ELEVENLABS_API_KEY;
const BASE_URL = 'https://api.elevenlabs.io/v1';

async function synthesize(text, voiceId = 'EXAVITQu4vr4xnSDxMaL') {
  const url = `${BASE_URL}/text-to-speech/${voiceId}`;
  const payload = {
    text,
    model_id: 'eleven_monolingual_v1',
    voice_settings: { stability: 0.75, similarity_boost: 0.8 }
  };
  const res = await fetch(url, {
    method: 'POST',
    headers: { 'xi-api-key': ELEVEN_API, 'Content-Type': 'application/json' },
    body: JSON.stringify(payload)
  });
  if (!res.ok) throw new Error(`ElevenLabs error: ${res.statusText}`);
  return await res.buffer();
}

async function buildVoicemail(data) {
  const template = `Hi ${data.name}, this is a reminder from ${data.company}. 
Your appointment is scheduled for ${data.date} at ${data.time}. 
If you need to reschedule, please call us at ${data.phone}. 
Thanks, and see you soon!`;
  const audioBuffer = await synthesize(template);
  const outPath = `voicemail_${data.name.replace(/\s+/g, '_')}.mp3`;
  fs.writeFileSync(outPath, audioBuffer);
  console.log(`Voicemail saved to ${outPath}`);
}

The core logic is identical in both versions; just choose the stack you're familiar with.

Optional Advanced: Voice Cloning and API Deployment

If brand consistency is a high priority, you can use ElevenLabs' voice cloning feature. The process is: POST audio samples to /v1/voices/add, wait for the cloning task to complete and get the voice_id, then replace the default ID in the synthesize call. The cloned voice will "sound exactly like the original speaker."

When the generator needs to be called by a CRM or other services, you can wrap it in a Flask or Express endpoint. In the Flask example, the /voicemail route receives JSON data, fills the template, and returns the MP3 stream directly via send_file. Deploy it to Render, Fly.io, or any VPS, and you have a RESTful voicemail generator.

Calling constraints of the /v1/voices/add endpoint
Voice cloning is not unlimited: ElevenLabs requires clear audio samples without background noise, and usually needs multiple files (or a single longer file) to cover different intonations. When calling /v1/voices/add, submit fields such as name and files as multipart/form-data; the response returns a new voice_id. Cloning tasks may complete asynchronously, so you need to poll or wait for a callback. In addition, the use of cloned voices is subject to account plans and consent terms—you must ensure the sample provider has authorized it. When deploying as a REST API, set request timeouts, rate limiting, and error retries, because TTS synthesis may be delayed by text length or service load.

Key Takeaways and Further Thoughts

The value of this project lies in abstracting complex speech synthesis into a pipeline: template rendering handles dynamic content, API calls handle voice quality, and audio post-processing and HTTP delivery handle integration. Since the audio fidelity returned by ElevenLabs is already high, most scenarios don't need extra DSP tricks; and the presence of ffmpeg means you can still meet production-level needs such as concatenation and volume normalization.

For teams needing multilingual support, simply switch the template language and pair it with a multilingual model to cover more users while maintaining a consistent brand voice.

One-line Summary

Build a voicemail generator fast with the ElevenLabs TTS API using template filling and Python/Node.js.

Paragraph Summary

This article explains how to quickly build a voicemail generator using ElevenLabs' text-to-speech API. The core workflow is filling a text template, calling the TTS interface, and saving the MP3, which can be embedded in webhooks, CI, or CLI tools. Preparation includes getting an API key and storing it in an environment variable, and designing a reusable template with dynamic variables. The Python version uses requests and pydub, while the Node.js version uses node-fetch and fluent-ffmpeg, with identical logic. The advanced section covers voice cloning (replacing the default timbre via /v1/voices/add) and wrapping the generator as a Flask or Express REST endpoint for CRM calls. The article emphasizes that ElevenLabs' free tier is enough for prototype validation and audio fidelity is high, usually requiring no extra DSP processing.