Add real-time voice conversations to a custom agent runtime with ElevenLabs Speech Engine. Use when building Speech Engine servers, WebSocket handlers, WebRTC browser clients, conversation token endpoints, interruption-aware streaming responses, or voice-enabled chat agents that connect developer-owned server logic to ElevenLabs speech-to-text and text-to-speech.
Detected risks:
The speech-engine skill adds real-time voice conversations to a custom agent runtime using the ElevenLabs Speech Engine. It solves the problem of voice-enabling a developer-owned server or chat app without handing agent logic to a hosted platform: ElevenLabs handles microphone audio, speech-to-text, turn-taking, text-to-speech, and browser playback, while your server exposes a Speech Engine WebSocket endpoint and streams response text back. Each WebSocket connection represents one conversation, and the SDK manages WebSocket routing, request verification, session lifecycle, ping/pong, turn-taking, and interruption handling.
The implementation flow includes installing server dependencies and configuring ELEVENLABS_API_KEY, exposing the server through a public HTTPS URL for local development (for example with ngrok http 3001), creating a Speech Engine resource with ws_url / wsUrl, storing the returned ID (for example in ELEVENLABS_SPEECH_ENGINE_ID), and starting the server with engine.serve(...) in Python or speechEngine.attach(...) / engine.attach(...) in TypeScript. Browser clients connect using a server-issued conversationToken, built with @elevenlabs/react or @elevenlabs/client, and the API key must never appear in browser code. sendResponse() / send_response() accepts a string or async iterable of response text. Create requests can also configure tts, asr, turn, request_headers, overrides, and privacy. Callbacks distinguish clean closes (onClose / on_close) from unexpected drops (onDisconnect / on_disconnect), and a first-message override lets the agent greet first.
The skill targets developers building voice-enabled chat agents, WebSocket handlers, WebRTC browser clients, or connecting runtimes like OpenClaw or Hermes to ElevenLabs speech. It requires internet access and an ElevenLabs API key. It stresses security: speech-recognition text is untrusted and prone to prompt injection, so it must be converted to trusted application state before driving responses or tools. Every WebSocket verifies a JWT by default; the disableAuth / disable_auth escape hatch should only be used with a compensating control such as an IP allowlist or a custom shared-secret header. For a hosted Conversational AI agent with platform-managed prompts and tools, the skill directs users to the separate agents skill.
ELEVENLABS_API_KEY is required, along with internet access. The skill also references ELEVENLABS_SPEECH_ENGINE_ID for storing the created engine's ID and a PUBLIC_WS_URL for the public WebSocket route.
Expose your Speech Engine server through a public HTTPS URL, for example with ngrok http 3001, then create the Speech Engine resource with ws_url / wsUrl pointing at the public WebSocket URL (usually wss://.../ws).
Issue conversation tokens from a server endpoint and start the client session with the conversationToken using @elevenlabs/react or @elevenlabs/client. Never put ELEVENLABS_API_KEY in browser code.
No, not by default. Every incoming WebSocket verifies a JWT to prove the connection comes from ElevenLabs. The disableAuth / disable_auth escape hatch should only be used when a compensating control such as an IP allowlist or a custom shared-secret header is already in place; otherwise any client that finds your URL can open sessions.
Treat it as untrusted input that may contain prompt-injection attempts. Use deterministic validation, allowlisted intents, or explicit user confirmation to convert it into trusted application state before it affects response generation, tools, or privileged actions.
Use the agents skill when creating or configuring a hosted ElevenLabs Conversational AI agent with platform-managed prompts, tools, workflows, phone numbers, or widgets. Use speech-engine when you want to keep agent logic on your own server.
Quick Setup:
.claude/skills/Repository
elevenlabs/skills