From 7806f11a8738f5ea8bfbd08919746f42322a2088 Mon Sep 17 00:00:00 2001 From: James Hush Date: Tue, 7 Jul 2026 15:39:20 +0800 Subject: [PATCH 1/4] Docs improvements from Kapa top questions analysis - Add 'Is Pipecat free?' notes to intro pages and a pricing link to the Pipecat Cloud intro (top question cluster: pricing and costs) - Add new Interruptions fundamentals page explaining barge-in mechanics, verified against the Pipecat source (cluster: interruptions and turn-taking) - Fix quickstart walkthrough snippets that drifted from the actual 'pipecat init quickstart' scaffold: OpenAIResponsesLLMService, RTVI on_client_ready greeting handler, runner.add_workers + run() --- docs.json | 1 + overview/introduction.mdx | 9 ++ pipecat-cloud/introduction.mdx | 2 + pipecat/fundamentals/interruptions.mdx | 126 +++++++++++++++++++++++++ pipecat/get-started/introduction.mdx | 8 ++ pipecat/get-started/quickstart.mdx | 27 +++--- 6 files changed, 161 insertions(+), 12 deletions(-) create mode 100644 pipecat/fundamentals/interruptions.mdx diff --git a/docs.json b/docs.json index b91ee7173..c95f94bfb 100644 --- a/docs.json +++ b/docs.json @@ -72,6 +72,7 @@ "group": "Fundamentals", "pages": [ "pipecat/fundamentals/service-settings", + "pipecat/fundamentals/interruptions", "pipecat/fundamentals/user-input-muting", "pipecat/fundamentals/detecting-user-idle", "pipecat/fundamentals/stt-latency-tuning", diff --git a/overview/introduction.mdx b/overview/introduction.mdx index 6639ffd3a..84d02097e 100644 --- a/overview/introduction.mdx +++ b/overview/introduction.mdx @@ -5,6 +5,15 @@ description: "Build voice and multimodal AI agents with the Pipecat ecosystem." Pipecat is an open source ecosystem for building voice and multimodal AI agents. It provides everything you need to create, deploy, and scale real-time AI applications that can see, hear, and speak. + + **Is Pipecat free?** Yes. The Pipecat framework is free and open source + (BSD-2 license), and there is no charge to use it. Your costs come from two + places: the AI services your agent calls (speech-to-text, LLM, and + text-to-speech providers bill you directly) and hosting, either your own + infrastructure or [Pipecat + Cloud](https://www.daily.co/pricing/pipecat-cloud). + + ## The Pipecat Ecosystem diff --git a/pipecat-cloud/introduction.mdx b/pipecat-cloud/introduction.mdx index cb2df907e..0095cfb17 100644 --- a/pipecat-cloud/introduction.mdx +++ b/pipecat-cloud/introduction.mdx @@ -5,6 +5,8 @@ description: "Deploy your AI agents to production at scale" [Pipecat Cloud](https://pipecat.daily.co) is a managed platform for deploying and scaling Pipecat agents in production. It handles infrastructure, scaling, and operations so you can focus on building your agent. +Pipecat Cloud uses usage-based pricing. See the [pricing page](https://www.daily.co/pricing/pipecat-cloud) for current rates and a calculator to estimate your total cost per minute, including third-party AI services. + diff --git a/pipecat/fundamentals/interruptions.mdx b/pipecat/fundamentals/interruptions.mdx new file mode 100644 index 000000000..768dbf737 --- /dev/null +++ b/pipecat/fundamentals/interruptions.mdx @@ -0,0 +1,126 @@ +--- +title: "Interruptions" +description: "How Pipecat stops the bot when the user speaks, what happens to in-flight LLM and TTS output, and how to control it." +--- + +Interruptions (also called barge-in) let the user talk over the bot. When the user starts speaking while the bot is talking, the bot stops immediately, in-flight work is cancelled, and the pipeline is ready for the new user input. + +Interruptions are **enabled by default**. This page explains how they work under the hood, what ends up in the conversation context, and how to configure or trigger them yourself. + + + Looking for how Pipecat decides *when* a user turn starts and ends? See + [Speech Input & Turn Detection](/pipecat/learn/speech-input) and [User Turn + Strategies](/api-reference/server/utilities/turn-management/user-turn-strategies). + This page covers what happens *after* an interruption is triggered. + + +## What happens when the user interrupts + +When a [user turn start strategy](/api-reference/server/utilities/turn-management/user-turn-strategies#start-strategies) triggers with `enable_interruptions=True` (the default), the user aggregator broadcasts an `InterruptionFrame` both upstream and downstream through the pipeline. From there: + + + + `InterruptionFrame` is a + [SystemFrame](/api-reference/server/frames/system-frames), so each processor + handles it immediately instead of waiting behind queued frames. Each + processor cancels its processing task and discards queued `DataFrame`s and + `ControlFrame`s. Frames marked uninterruptible (like + `FunctionCallResultFrame` and `EndFrame`) are preserved and still processed. + + + The in-flight LLM completion is cancelled mid-stream. Any registered + function calls with `cancel_on_interruption=True` are cancelled and emit a + `FunctionCallCancelFrame`. See [Function + Calling](/pipecat/learn/function-calling) for details. + + + The TTS service stops synthesizing, clears its text aggregation and word + timestamps, and drops pending output. + + + The output transport drains its audio queue, discarding audio that was + generated but not yet played. If a background audio mixer is active, the + transport drains only the bot's speech so the background audio keeps playing + without a gap. + + + +The result: the bot goes silent within roughly one audio write, and the pipeline is clean and ready for the user's new turn. + +## What ends up in the context + +A common question: if the bot is cut off mid-sentence, what does the LLM context contain? + +**Only the words that were actually spoken.** As the bot speaks, the output transport pushes `TTSTextFrame`s downstream in sync with audio playback. Text that never played never reaches the assistant context aggregator. On interruption, the aggregator commits the partial, spoken-so-far text to the context as the assistant message. + +This means the LLM's next completion sees an accurate transcript of the conversation: the bot's message ends where the user cut it off, not where the LLM's generation ended. + +You can observe this with the `on_assistant_turn_stopped` event, which reports the committed text and whether the turn was interrupted: + +```python +@assistant_aggregator.event_handler("on_assistant_turn_stopped") +async def on_assistant_turn_stopped(aggregator, message): + if message.interrupted: + print(f"Bot was cut off after saying: {message.content}") +``` + +## Controlling interruptions + +Interruption behavior is configured on the [user turn start strategies](/api-reference/server/utilities/turn-management/user-turn-strategies#start-strategies): + +**Disable interruptions** so the bot always finishes speaking: + +```python +from pipecat.turns.user_start import VADUserTurnStartStrategy + +start_strategy = VADUserTurnStartStrategy(enable_interruptions=False) +``` + +**Require a minimum number of words** so short utterances like "okay" or "yeah" don't interrupt the bot: + +```python +from pipecat.turns.user_start import MinWordsUserTurnStartStrategy + +start_strategy = MinWordsUserTurnStartStrategy(min_words=3) +``` + +**Filter out backchannels with a model.** The [Krisp VIVA Interruption Prediction strategy](/api-reference/server/utilities/turn-management/user-turn-strategies#krispvivaipuserturnstartstrategy) distinguishes genuine interruptions from acknowledgments like "uh-huh". + +**Mute user input entirely** during certain phases (like the bot's introduction) with [user input muting](/pipecat/fundamentals/user-input-muting). + +## Triggering an interruption yourself + +Sometimes the bot should stop itself: a timeout fires, an external event arrives, or your own logic decides the current response is no longer relevant. + +From inside a custom `FrameProcessor`, broadcast the interruption directly: + +```python +await self.broadcast_interruption() +``` + +From code that has a reference to a processor or the worker, you can also push an `InterruptionWorkerFrame`. The pipeline worker converts it into an `InterruptionFrame` and sends it through the whole pipeline: + +```python +from pipecat.frames.frames import InterruptionWorkerFrame + +await worker.queue_frame(InterruptionWorkerFrame()) +``` + +Both approaches run the same interruption flow described above: the bot stops speaking, in-flight work is cancelled, and the spoken-so-far text is committed to context. + + + `InterruptionWorkerFrame` was called `BotInterruptionFrame` before Pipecat + 1.0. See the [migration guide](/pipecat/migration/migration-1.0) if you're + upgrading older code. + + +## Speech-to-speech services + +Realtime speech-to-speech services (like Gemini Live and OpenAI Realtime) handle interruption detection on the provider side. In these pipelines, the service emits the speaking events and Pipecat's aggregators follow along using [external turn strategies](/api-reference/server/utilities/turn-management/user-turn-strategies#externaluserturnstartstrategy). The provider decides when the user barged in; Pipecat still flushes local audio output so the bot goes silent right away. + +## Related + +- [Speech Input & Turn Detection](/pipecat/learn/speech-input) - how user turns are detected +- [User Turn Strategies](/api-reference/server/utilities/turn-management/user-turn-strategies) - full strategy reference +- [User Input Muting](/pipecat/fundamentals/user-input-muting) - suppress user input while the bot speaks +- [System Frames](/api-reference/server/frames/system-frames) - `InterruptionFrame` reference diff --git a/pipecat/get-started/introduction.mdx b/pipecat/get-started/introduction.mdx index cfd6aa234..17ab1df9b 100644 --- a/pipecat/get-started/introduction.mdx +++ b/pipecat/get-started/introduction.mdx @@ -5,6 +5,14 @@ description: "Open source Python framework for building voice and multimodal AI Pipecat is an open source Python framework for building voice and multimodal AI agents. It orchestrates AI services, network transports, and audio processing to enable ultra-low latency conversations that feel natural and responsive. + + **Is Pipecat free?** Yes. The framework is free and open source (BSD-2 + license). You only pay for the AI services your agent uses (speech-to-text, + LLM, and text-to-speech providers bill you directly) and for hosting: your + own infrastructure or [Pipecat + Cloud](https://www.daily.co/pricing/pipecat-cloud). + + Want to dive right in? Build and run your first Pipecat application diff --git a/pipecat/get-started/quickstart.mdx b/pipecat/get-started/quickstart.mdx index 4447521ab..6c67f17e5 100644 --- a/pipecat/get-started/quickstart.mdx +++ b/pipecat/get-started/quickstart.mdx @@ -225,11 +225,11 @@ tts = CartesiaTTSService( voice=os.getenv("CARTESIA_VOICE_ID", "71a7ad14-091c-4e8e-a314-022ece01c121"), ), ) -llm = OpenAIRealtimeLLMService( +llm = OpenAIResponsesLLMService( api_key=os.getenv("OPENAI_API_KEY"), - settings=OpenAIRealtimeLLMService.Settings( + settings=OpenAIResponsesLLMService.Settings( model=os.getenv("OPENAI_MODEL", "gpt-4.1"), - system_instruction="You are a helpful assistant.", + system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.", ), ) ``` @@ -297,25 +297,27 @@ The worker handles pipeline execution, collects metrics, and manages RTVI events Event handlers manage the bot's lifecycle and user interactions: ```python +# Kick off the conversation when the client signals it's ready +@worker.rtvi.event_handler("on_client_ready") +async def on_client_ready(rtvi): + context.add_message({"role": "developer", "content": "Start by concisely introducing yourself."}) + await worker.queue_frames([LLMRunFrame()]) + # Event handler for when a client connects @transport.event_handler("on_client_connected") async def on_client_connected(transport, client): - logger.info(f"Client connected") - # Add a greeting message to the context - context.add_message("developer", "Say hello and briefly introduce yourself.") - # Prompt the bot to start talking when the client connects - await worker.queue_frames([LLMRunFrame()]) + logger.info("Client connected") # Event handler for when a client disconnects @transport.event_handler("on_client_disconnected") async def on_client_disconnected(transport, client): - logger.info(f"Client disconnected") + logger.info("Client disconnected") # Cancel the worker when the client disconnects # This stops the pipeline and all processors, cleaning up resources await worker.cancel() ``` -When a client connects, the bot adds a greeting instruction and queues a context frame to initiate the conversation. When disconnecting, it properly cancels the worker to clean up resources. +When the client signals it's ready over the RTVI protocol, the bot adds a greeting instruction and queues an `LLMRunFrame` to start the conversation. When the client disconnects, the worker is cancelled to clean up resources. ### Running the Pipeline @@ -325,9 +327,10 @@ Finally, the pipeline is executed by a WorkerRunner: # Create a WorkerRunner to run the worker runner = WorkerRunner(handle_sigint=False) -# Finally, run the worker using the runner +# Finally, add the worker and run it # This will start the pipeline and begin processing frames -await runner.run(worker) +await runner.add_workers(worker) +await runner.run() ``` The runner manages the pipeline's execution lifecycle. Note that `handle_sigint=False` because the main runner handles system signals. From c3f7b3a46e0aaf0f3f9d806ec7729625a28ae44c Mon Sep 17 00:00:00 2001 From: James Hush Date: Tue, 7 Jul 2026 15:45:17 +0800 Subject: [PATCH 2/4] Remove BotInterruptionFrame migration note from interruptions page --- pipecat/fundamentals/interruptions.mdx | 6 ------ 1 file changed, 6 deletions(-) diff --git a/pipecat/fundamentals/interruptions.mdx b/pipecat/fundamentals/interruptions.mdx index 768dbf737..b46a2e973 100644 --- a/pipecat/fundamentals/interruptions.mdx +++ b/pipecat/fundamentals/interruptions.mdx @@ -108,12 +108,6 @@ await worker.queue_frame(InterruptionWorkerFrame()) Both approaches run the same interruption flow described above: the bot stops speaking, in-flight work is cancelled, and the spoken-so-far text is committed to context. - - `InterruptionWorkerFrame` was called `BotInterruptionFrame` before Pipecat - 1.0. See the [migration guide](/pipecat/migration/migration-1.0) if you're - upgrading older code. - - ## Speech-to-speech services Realtime speech-to-speech services (like Gemini Live and OpenAI Realtime) handle interruption detection on the provider side. In these pipelines, the service emits the speaking events and Pipecat's aggregators follow along using [external turn strategies](/api-reference/server/utilities/turn-management/user-turn-strategies#externaluserturnstartstrategy). The provider decides when the user barged in; Pipecat still flushes local audio output so the bot goes silent right away. From 9e893af33a516e641366a383e37f5bfb6d79751f Mon Sep 17 00:00:00 2001 From: Mark Backman Date: Thu, 16 Jul 2026 14:37:00 -0400 Subject: [PATCH 3/4] Update interruptions guide, cross-link --- pipecat/fundamentals/interruptions.mdx | 37 ++++++++++++++++++++------ pipecat/learn/context-management.mdx | 4 ++- pipecat/learn/speech-input.mdx | 21 ++------------- 3 files changed, 34 insertions(+), 28 deletions(-) diff --git a/pipecat/fundamentals/interruptions.mdx b/pipecat/fundamentals/interruptions.mdx index b46a2e973..1b5f43da7 100644 --- a/pipecat/fundamentals/interruptions.mdx +++ b/pipecat/fundamentals/interruptions.mdx @@ -66,17 +66,24 @@ async def on_assistant_turn_stopped(aggregator, message): ## Controlling interruptions -Interruption behavior is configured on the [user turn start strategies](/api-reference/server/utilities/turn-management/user-turn-strategies#start-strategies): - -**Disable interruptions** so the bot always finishes speaking: +**Let the bot finish speaking** with [user input muting](/pipecat/fundamentals/user-input-muting). Mute strategies block user audio, transcriptions, and interruption signals while active, so speech over the bot is discarded rather than answered later. Use `AlwaysUserMuteStrategy` to mute whenever the bot is speaking, or `FirstSpeechUserMuteStrategy` to protect just the introduction: ```python -from pipecat.turns.user_start import VADUserTurnStartStrategy - -start_strategy = VADUserTurnStartStrategy(enable_interruptions=False) +from pipecat.processors.aggregators.llm_response_universal import ( + LLMContextAggregatorPair, + LLMUserAggregatorParams, +) +from pipecat.turns.user_mute import AlwaysUserMuteStrategy + +user_aggregator, assistant_aggregator = LLMContextAggregatorPair( + context, + user_params=LLMUserAggregatorParams( + user_mute_strategies=[AlwaysUserMuteStrategy()], + ), +) ``` -**Require a minimum number of words** so short utterances like "okay" or "yeah" don't interrupt the bot: +**Require a minimum number of words** so short utterances like "okay" or "yeah" don't interrupt the bot. This is configured on the [user turn start strategy](/api-reference/server/utilities/turn-management/user-turn-strategies#start-strategies): ```python from pipecat.turns.user_start import MinWordsUserTurnStartStrategy @@ -86,7 +93,21 @@ start_strategy = MinWordsUserTurnStartStrategy(min_words=3) **Filter out backchannels with a model.** The [Krisp VIVA Interruption Prediction strategy](/api-reference/server/utilities/turn-management/user-turn-strategies#krispvivaipuserturnstartstrategy) distinguishes genuine interruptions from acknowledgments like "uh-huh". -**Mute user input entirely** during certain phases (like the bot's introduction) with [user input muting](/pipecat/fundamentals/user-input-muting). +**Disable interruptions** so in-flight work is never cancelled: + +```python +from pipecat.turns.user_start import VADUserTurnStartStrategy + +start_strategy = VADUserTurnStartStrategy(enable_interruptions=False) +``` + + + Disabling interruptions does not ignore the user. Speech over the bot is + still transcribed and processed as a normal user turn — the bot's reply is + queued and plays as soon as the current speech finishes. If you want the bot + to finish speaking *and* discard what the user said over it, use mute + strategies instead. + ## Triggering an interruption yourself diff --git a/pipecat/learn/context-management.mdx b/pipecat/learn/context-management.mdx index 04bf3e846..9bd80a081 100644 --- a/pipecat/learn/context-management.mdx +++ b/pipecat/learn/context-management.mdx @@ -215,7 +215,9 @@ Place the assistant context aggregator **after `transport.output()`**. This posi Always place the assistant context aggregator **after** `transport.output()` - to ensure proper word-level context updates during interruptions. + to ensure proper word-level context updates during interruptions. See + [Interruptions](/pipecat/fundamentals/interruptions#what-ends-up-in-the-context) + for what the context contains when the bot is cut off mid-sentence. ## Manual Context Control diff --git a/pipecat/learn/speech-input.mdx b/pipecat/learn/speech-input.mdx index eee04be6d..196bbe830 100644 --- a/pipecat/learn/speech-input.mdx +++ b/pipecat/learn/speech-input.mdx @@ -156,26 +156,9 @@ stop_strategy = SpeechTimeoutUserTurnStopStrategy(user_speech_timeout=0.6) ### Interruptions -Interruptions stop the bot when the user starts speaking. This is controlled by the `enable_interruptions` parameter on start strategies (enabled by default). +When a user turn starts with interruptions enabled (the default, via the `enable_interruptions` parameter on start strategies), the bot immediately stops speaking, in-flight work is cancelled, and the pipeline is ready for the new user input. -When a user turn starts with interruptions enabled: - -1. Bot immediately stops speaking -2. Pending audio and text is cleared -3. Pipeline ready for new user input - -To disable interruptions: - -```python -from pipecat.turns.user_start import VADUserTurnStartStrategy - -start_strategy = VADUserTurnStartStrategy(enable_interruptions=False) -``` - - - Keep interruptions enabled (default) for natural conversations. This enables - users to interrupt the bot mid-sentence, just like human conversations. - +Keep interruptions enabled for natural conversations. For how the cancellation works, what ends up in the LLM context, and how to tune, disable, or trigger interruptions yourself, see [Interruptions](/pipecat/fundamentals/interruptions). ## Best Practices From e1109899558b89fb75875ea0970df103b84aec9b Mon Sep 17 00:00:00 2001 From: Mark Backman Date: Thu, 16 Jul 2026 14:47:39 -0400 Subject: [PATCH 4/4] Reframe pricing notes as flexibility, move Cloud pricing to its own section MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replace the "Is Pipecat free?" Note callouts on the ecosystem and framework intro pages with a sentence in the opening paragraph: free under BSD-2, bring any provider, pay only for the services you choose. On the Pipecat Cloud intro, move the usage-based pricing paragraph out of the page opening into a dedicated Pricing section before Next Steps. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- overview/introduction.mdx | 11 +---------- pipecat-cloud/introduction.mdx | 6 ++++-- pipecat/get-started/introduction.mdx | 10 +--------- 3 files changed, 6 insertions(+), 21 deletions(-) diff --git a/overview/introduction.mdx b/overview/introduction.mdx index 84d02097e..d5b272fec 100644 --- a/overview/introduction.mdx +++ b/overview/introduction.mdx @@ -3,16 +3,7 @@ title: "Introduction" description: "Build voice and multimodal AI agents with the Pipecat ecosystem." --- -Pipecat is an open source ecosystem for building voice and multimodal AI agents. It provides everything you need to create, deploy, and scale real-time AI applications that can see, hear, and speak. - - - **Is Pipecat free?** Yes. The Pipecat framework is free and open source - (BSD-2 license), and there is no charge to use it. Your costs come from two - places: the AI services your agent calls (speech-to-text, LLM, and - text-to-speech providers bill you directly) and hosting, either your own - infrastructure or [Pipecat - Cloud](https://www.daily.co/pricing/pipecat-cloud). - +Pipecat is an open source ecosystem for building voice and multimodal AI agents. It provides everything you need to create, deploy, and scale real-time AI applications that can see, hear, and speak. The framework is free to use under the BSD-2 license, and works with any AI provider and any hosting environment — you pay only for the services you choose. ## The Pipecat Ecosystem diff --git a/pipecat-cloud/introduction.mdx b/pipecat-cloud/introduction.mdx index 0095cfb17..e84f886b3 100644 --- a/pipecat-cloud/introduction.mdx +++ b/pipecat-cloud/introduction.mdx @@ -5,8 +5,6 @@ description: "Deploy your AI agents to production at scale" [Pipecat Cloud](https://pipecat.daily.co) is a managed platform for deploying and scaling Pipecat agents in production. It handles infrastructure, scaling, and operations so you can focus on building your agent. -Pipecat Cloud uses usage-based pricing. See the [pricing page](https://www.daily.co/pricing/pipecat-cloud) for current rates and a calculator to estimate your total cost per minute, including third-party AI services. - @@ -51,6 +49,10 @@ Pipecat Cloud uses usage-based pricing. See the [pricing page](https://www.daily +## Pricing + +Pipecat Cloud uses usage-based pricing. See the [pricing page](https://www.daily.co/pricing/pipecat-cloud) for current rates and a calculator to estimate your total cost per minute, including third-party AI services. + ## Next Steps diff --git a/pipecat/get-started/introduction.mdx b/pipecat/get-started/introduction.mdx index 17ab1df9b..d67e92016 100644 --- a/pipecat/get-started/introduction.mdx +++ b/pipecat/get-started/introduction.mdx @@ -3,15 +3,7 @@ title: "Pipecat" description: "Open source Python framework for building voice and multimodal AI agents." --- -Pipecat is an open source Python framework for building voice and multimodal AI agents. It orchestrates AI services, network transports, and audio processing to enable ultra-low latency conversations that feel natural and responsive. - - - **Is Pipecat free?** Yes. The framework is free and open source (BSD-2 - license). You only pay for the AI services your agent uses (speech-to-text, - LLM, and text-to-speech providers bill you directly) and for hosting: your - own infrastructure or [Pipecat - Cloud](https://www.daily.co/pricing/pipecat-cloud). - +Pipecat is an open source Python framework for building voice and multimodal AI agents. It orchestrates AI services, network transports, and audio processing to enable ultra-low latency conversations that feel natural and responsive. The framework is free to use under the BSD-2 license, and you choose your own stack: any speech-to-text, LLM, and text-to-speech provider, hosted on your own infrastructure or [Pipecat Cloud](https://www.daily.co/pricing/pipecat-cloud) — you pay only for the providers you choose. Want to dive right in? Build and run your first Pipecat application