fix(0.6.1): pipeline firstMessage barge-in aborts TTS stream + unblocks LLM#100
Open
nicolotognoni wants to merge 1 commit into
Conversation
…ks LLM When the user interrupted the agent during the firstMessage in pipeline mode (Deepgram STT + LLM + ElevenLabs WS TTS), the existing barge-in cancel flipped ``isSpeaking`` / ``_is_speaking`` to ``False`` but the ``for await`` / ``async for`` consuming ``tts.synthesizeStream`` / ``tts.synthesize`` stayed suspended on the next-frame wait (``ws.recv()``). The check at the top of the loop body never re-ran, the provider WS sat idle until ``FRAME_TIMEOUT_MS`` (30 s on ElevenLabs WS TTS), and the "speaking lock" was never released — subsequent Deepgram finals were captured but the LLM dispatch path never fired, leaving the call silent for the user. Fix (parity Py/TS): * Add ``firstMessageAbort`` (TS, ``AbortController``) / ``_first_message_abort`` (Py, ``asyncio.Event``) raced against the iterator's ``next()`` / ``__anext__``. * Add an optional ``cancel()`` hook on the TTS adapter interface. Implementation in ``ElevenLabsWebSocketTTS`` (both SDKs) closes the in-flight WS (``activeSocket`` / ``_active_socket``) so the next-frame wait unblocks via ``ConnectionClosed`` within one event-loop tick. * Use the manual iterator protocol in the firstMessage loop so we can race ``next()`` with the abort signal and call ``iter.return()`` / ``agen.aclose()`` on abort, ensuring the generator's ``finally`` runs and closes the WS. * Regression tests in both SDKs: standalone iterator race + ``_handle_barge_in`` must invoke ``tts.cancel`` when available and set the abort event. CHANGELOG entry added under ``## 0.6.1 (2026-05-12)``.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
cancelSpeakingflippedisSpeaking=falsebut thefor awaitovertts.synthesizeStreamstayed suspended on the provider WS untilFRAME_TIMEOUT_MS(30 s on ElevenLabs WS), so no subsequent LLM turn ever dispatched even though Deepgram kept transcribing.next()/__anext__plus an optionalTTSAdapter.cancel()hook that closes the in-flight provider WS, so the firstMessage loop exits within one event-loop tick on barge-in.firstMessageAbort/_first_message_abort,activeSocket/_active_socket,cancel()on bothElevenLabsWebSocketTTSimplementations.Implementation
libraries/typescript/src/stream-handler.tsfirstMessageAbort: AbortController | null.cancelSpeaking()now also callsfirstMessageAbort.abort().for awaitwith a manual async-iterator loop racingiter.next()against the abort signal; on abort callsthis.tts?.cancel?.()then awaitsiter.return()to drive the generator'sfinally.libraries/typescript/src/provider-factory.ts— added optionalcancel?()method onTTSAdapter.libraries/typescript/src/providers/elevenlabs-ws-tts.ts— trackactiveSocketacrosssynthesizeStream, addcancel()that closes it, foldcancel()intoclose().libraries/python/getpatter/stream_handler.pyself._first_message_abort: asyncio.Event | None._do_cancel_for_barge_insets the abort event and callstts.cancel()when the adapter exposes it.async forwith a manualasyncio.waitrace of__anext__againstfm_abort.wait();agen.aclose()always runs in finally.libraries/python/getpatter/providers/elevenlabs_ws_tts.py— trackself._active_socketacrosssynthesize, addcancel()that schedulesws.close()on the loop, catchConnectionClosedin the recv loop as end-of-stream._handle_barge_insets abort event + invokestts.cancelwhen available, plus a standalone iterator-race test that proves a stalled generator unblocks within one tick of (abort + adapter.cancel).## 0.6.1### Fixedentry.Breaking change?
No. The new
TTSAdapter.cancel()is optional — existing adapters without it are unaffected (the stream handler null-checks before calling). All existing public API shapes are preserved.Test plan
pytest tests/— 1845 passed, 7 skipped, 0 failed.npx vitest run— 1518 passed, 0 failed.npm run lint(tsc --noEmit) — clean.npx tsupbuild — success.ElevenLabsWebSocketTTS, interrupt the agent ~500 ms into the firstMessage, confirm the next user turn dispatches an LLM response (no 30 s silence, noFRAME_TIMEOUT_MSerror).Docs updates
N/A — internal fix, no public API surface change beyond the optional
cancel()hook (which is documented inline in theTTSAdapterinterface).