ElevenLabs Core Workflow B — Speech-to-Speech, Sound Effects & Audio Isolation
Overview
Secondary ElevenLabs workflows beyond TTS: (1) Speech-to-Speech voice conversion, (2) Sound Effects generation from text descriptions, (3) Audio Isolation for noise removal, and (4) Speech-to-Text transcription. Each maps to one API endpoint and has both a TypeScript SDK and a cURL path.
Full code for every step lives in references/implementation.md; copy-ready invocations are in references/examples.md.
Prerequisites
- Completed
elevenlabs-install-authsetup. - For STS: source audio file in MP3/WAV/M4A format.
- For audio isolation: noisy audio file to clean.
Authentication
The SDK client (new ElevenLabsClient()) reads the API key from the
ELEVENLABS_API_KEY environment variable automatically — never hardcode it. cURL
requests send it as the xi-api-key: ${ELEVENLABS_API_KEY} header. Full auth setup
is covered by the elevenlabs-install-auth skill.
Instructions
Import the SDK once, then call the relevant module. The client authenticates from the environment:
import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";
import { createReadStream, createWriteStream } from "fs";
import { Readable } from "stream";
import { pipeline } from "stream/promises";
const client = new ElevenLabsClient();
- Speech-to-Speech (voice changer) —
client.speechToSpeech.convert(voiceId, …)againstPOST /v1/speech-to-speech/{voice_id}. Usemodel_id: "eleven_english_sts_v2"and setremove_background_noise: truefor built-in cleanup. - Sound Effects —
client.textToSoundEffects.convert({ text, … })againstPOST /v1/sound-generation. Tuneduration_seconds(0.5–30) andprompt_influence(0–1; higher follows the prompt more closely). - Audio Isolation —
client.audioIsolation.audioIsolation({ audio })againstPOST /v1/audio-isolation, or the streaming variant for large files. - Speech-to-Text —
client.speechToText.convert({ audio, model_id: "scribe_v1" })againstPOST /v1/speech-to-text; optionally enablediarizeand word timestamps.
Each returns an audio stream (steps 1–3) piped to disk, or a transcript object (step 4). See references/implementation.md for the complete helper functions and cURL equivalents.
First example — Speech-to-Speech skeleton
async function speechToSpeech(sourceAudioPath, targetVoiceId, outputPath) {
const audio = await client.speechToSpeech.convert(targetVoiceId, {
audio: createReadStream(sourceAudioPath),
model_id: "eleven_english_sts_v2",
voice_settings: JSON.stringify({ stability: 0.5, similarity_boost: 0.8 }),
remove_background_noise: true,
});
await pipeline(Readable.fromWeb(audio as any), createWriteStream(outputPath));
}
API Endpoint Summary
| Feature | Method | Endpoint | Billing |
|---------|--------|----------|---------|
| Speech-to-Speech | POST | /v1/speech-to-speech/{voice_id} | Per character |
| Sound Effects | POST | /v1/sound-generation | Per generation |
| Audio Isolation | POST | /v1/audio-isolation | 1,000 chars/min of audio |
| Audio Isolation Stream | POST | /v1/audio-isolation/stream | 1,000 chars/min of audio |
| Speech-to-Text | POST | /v1/speech-to-text | Per audio minute |
Output
- Steps 1–3 write an audio file to the
outputPathyou pass and log a confirmation, e.g.Voice-converted audio saved to converted.mp3orClean audio saved to clean_interview.mp3. - Step 4 returns a transcript object:
result.textholds the full transcription, andresult.words(when present) carries word-level{ start, end, text }timestamps. - cURL paths stream the resulting audio directly to the
--outputfile.
Error Handling
| Error | HTTP | Cause | Solution |
|-------|------|-------|----------|
| model_can_not_do_voice_conversion | 400 | Wrong model for STS | Use eleven_english_sts_v2 |
| audio_too_short | 400 | STS input under 1 second | Use longer audio clip |
| audio_too_long | 400 | STS input over limit | Trim to under 5 minutes |
| invalid_sound_prompt | 400 | Nonsensical SFX description | Write descriptive, specific prompts |
| file_too_large | 413 | Audio isolation over 500MB | Compress or split the file |
| quota_exceeded | 401 | Character/generation limit hit | Check usage dashboard |
Examples
Worked, copy-ready invocations for all four workflows — including the three sound-effect variants (rain, laser, seamless forest loop), the "Rachel" voice conversion, an audio-isolation clean-up, and a transcription with word timestamps — are in references/examples.md. A one-liner:
// Generate a 10-second rain sound effect, faithful to the prompt
await generateSoundEffect(
"Heavy rain on a tin roof with distant thunder",
"rain.mp3",
{ duration: 10, promptInfluence: 0.6 }
);
Resources
- Full implementation walkthrough — every step's SDK + cURL code, sound-effect tips, and audio-isolation limits.
- Worked examples — copy-ready invocations per workflow.
- Speech-to-Speech API
- Sound Effects API
- Audio Isolation API
- Speech-to-Text API
Next Steps
For common errors, see elevenlabs-common-errors. For SDK patterns, see elevenlabs-sdk-patterns.