Agent Skills: ElevenLabs Core Workflow B — Speech-to-Speech, Sound Effects & Audio Isolation

ElevenLabs Core Workflow B — Speech-to-Speech, Sound Effects & Audio Isolation

Overview

Secondary ElevenLabs workflows beyond TTS: (1) Speech-to-Speech voice conversion, (2) Sound Effects generation from text descriptions, (3) Audio Isolation for noise removal, and (4) Speech-to-Text transcription. Each maps to one API endpoint and has both a TypeScript SDK and a cURL path.

Full code for every step lives in references/implementation.md; copy-ready invocations are in references/examples.md.

Prerequisites

• Completed elevenlabs-install-auth setup.
• For STS: source audio file in MP3/WAV/M4A format.
• For audio isolation: noisy audio file to clean.

Authentication

The SDK client (new ElevenLabsClient()) reads the API key from the ELEVENLABS_API_KEY environment variable automatically — never hardcode it. cURL requests send it as the xi-api-key: ${ELEVENLABS_API_KEY} header. Full auth setup is covered by the elevenlabs-install-auth skill.

Instructions

Import the SDK once, then call the relevant module. The client authenticates from the environment:

import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";
import { createReadStream, createWriteStream } from "fs";
import { Readable } from "stream";
import { pipeline } from "stream/promises";

const client = new ElevenLabsClient();

1. Speech-to-Speech (voice changer) — client.speechToSpeech.convert(voiceId, …) against POST /v1/speech-to-speech/{voice_id}. Use model_id: "eleven_english_sts_v2" and set remove_background_noise: true for built-in cleanup.
2. Sound Effects — client.textToSoundEffects.convert({ text, … }) against POST /v1/sound-generation. Tune duration_seconds (0.5–30) and prompt_influence (0–1; higher follows the prompt more closely).
3. Audio Isolation — client.audioIsolation.audioIsolation({ audio }) against POST /v1/audio-isolation, or the streaming variant for large files.
4. Speech-to-Text — client.speechToText.convert({ audio, model_id: "scribe_v1" }) against POST /v1/speech-to-text; optionally enable diarize and word timestamps.

Each returns an audio stream (steps 1–3) piped to disk, or a transcript object (step 4). See references/implementation.md for the complete helper functions and cURL equivalents.

First example — Speech-to-Speech skeleton

async function speechToSpeech(sourceAudioPath, targetVoiceId, outputPath) {
  const audio = await client.speechToSpeech.convert(targetVoiceId, {
    audio: createReadStream(sourceAudioPath),
    model_id: "eleven_english_sts_v2",
    voice_settings: JSON.stringify({ stability: 0.5, similarity_boost: 0.8 }),
    remove_background_noise: true,
  });
  await pipeline(Readable.fromWeb(audio as any), createWriteStream(outputPath));
}

API Endpoint Summary

| Feature | Method | Endpoint | Billing | |---------|--------|----------|---------| | Speech-to-Speech | POST | /v1/speech-to-speech/{voice_id} | Per character | | Sound Effects | POST | /v1/sound-generation | Per generation | | Audio Isolation | POST | /v1/audio-isolation | 1,000 chars/min of audio | | Audio Isolation Stream | POST | /v1/audio-isolation/stream | 1,000 chars/min of audio | | Speech-to-Text | POST | /v1/speech-to-text | Per audio minute |

Output

• Steps 1–3 write an audio file to the outputPath you pass and log a confirmation, e.g. Voice-converted audio saved to converted.mp3 or Clean audio saved to clean_interview.mp3.
• Step 4 returns a transcript object: result.text holds the full transcription, and result.words (when present) carries word-level { start, end, text } timestamps.
• cURL paths stream the resulting audio directly to the --output file.

Error Handling

| Error | HTTP | Cause | Solution | |-------|------|-------|----------| | model_can_not_do_voice_conversion | 400 | Wrong model for STS | Use eleven_english_sts_v2 | | audio_too_short | 400 | STS input under 1 second | Use longer audio clip | | audio_too_long | 400 | STS input over limit | Trim to under 5 minutes | | invalid_sound_prompt | 400 | Nonsensical SFX description | Write descriptive, specific prompts | | file_too_large | 413 | Audio isolation over 500MB | Compress or split the file | | quota_exceeded | 401 | Character/generation limit hit | Check usage dashboard |

Examples

Worked, copy-ready invocations for all four workflows — including the three sound-effect variants (rain, laser, seamless forest loop), the "Rachel" voice conversion, an audio-isolation clean-up, and a transcription with word timestamps — are in references/examples.md. A one-liner:

// Generate a 10-second rain sound effect, faithful to the prompt
await generateSoundEffect(
  "Heavy rain on a tin roof with distant thunder",
  "rain.mp3",
  { duration: 10, promptInfluence: 0.6 }
);

Resources

• Full implementation walkthrough — every step's SDK + cURL code, sound-effect tips, and audio-isolation limits.
• Worked examples — copy-ready invocations per workflow.
• Speech-to-Speech API
• Sound Effects API
• Audio Isolation API
• Speech-to-Text API

Next Steps

For common errors, see elevenlabs-common-errors. For SDK patterns, see elevenlabs-sdk-patterns.