Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs.json
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@
"group": "Fundamentals",
"pages": [
"pipecat/fundamentals/service-settings",
"pipecat/fundamentals/interruptions",
"pipecat/fundamentals/user-input-muting",
"pipecat/fundamentals/detecting-user-idle",
"pipecat/fundamentals/stt-latency-tuning",
Expand Down
2 changes: 1 addition & 1 deletion overview/introduction.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: "Introduction"
description: "Build voice and multimodal AI agents with the Pipecat ecosystem."
---

Pipecat is an open source ecosystem for building voice and multimodal AI agents. It provides everything you need to create, deploy, and scale real-time AI applications that can see, hear, and speak.
Pipecat is an open source ecosystem for building voice and multimodal AI agents. It provides everything you need to create, deploy, and scale real-time AI applications that can see, hear, and speak. The framework is free to use under the BSD-2 license, and works with any AI provider and any hosting environment — you pay only for the services you choose.

## The Pipecat Ecosystem

Expand Down
4 changes: 4 additions & 0 deletions pipecat-cloud/introduction.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,10 @@ description: "Deploy your AI agents to production at scale"
</Step>
</Steps>

## Pricing

Pipecat Cloud uses usage-based pricing. See the [pricing page](https://www.daily.co/pricing/pipecat-cloud) for current rates and a calculator to estimate your total cost per minute, including third-party AI services.

## Next Steps

<CardGroup cols={2}>
Expand Down
141 changes: 141 additions & 0 deletions pipecat/fundamentals/interruptions.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
---
title: "Interruptions"
description: "How Pipecat stops the bot when the user speaks, what happens to in-flight LLM and TTS output, and how to control it."
---

Interruptions (also called barge-in) let the user talk over the bot. When the user starts speaking while the bot is talking, the bot stops immediately, in-flight work is cancelled, and the pipeline is ready for the new user input.

Interruptions are **enabled by default**. This page explains how they work under the hood, what ends up in the conversation context, and how to configure or trigger them yourself.

<Tip>
Looking for how Pipecat decides *when* a user turn starts and ends? See
[Speech Input & Turn Detection](/pipecat/learn/speech-input) and [User Turn
Strategies](/api-reference/server/utilities/turn-management/user-turn-strategies).
This page covers what happens *after* an interruption is triggered.
</Tip>

## What happens when the user interrupts

When a [user turn start strategy](/api-reference/server/utilities/turn-management/user-turn-strategies#start-strategies) triggers with `enable_interruptions=True` (the default), the user aggregator broadcasts an `InterruptionFrame` both upstream and downstream through the pipeline. From there:

<Steps>
<Step title="Every processor cancels its current work">
`InterruptionFrame` is a
[SystemFrame](/api-reference/server/frames/system-frames), so each processor
handles it immediately instead of waiting behind queued frames. Each
processor cancels its processing task and discards queued `DataFrame`s and
`ControlFrame`s. Frames marked uninterruptible (like
`FunctionCallResultFrame` and `EndFrame`) are preserved and still processed.
</Step>
<Step title="The LLM stops generating">
The in-flight LLM completion is cancelled mid-stream. Any registered
function calls with `cancel_on_interruption=True` are cancelled and emit a
`FunctionCallCancelFrame`. See [Function
Calling](/pipecat/learn/function-calling) for details.
</Step>
<Step title="TTS clears its buffers">
The TTS service stops synthesizing, clears its text aggregation and word
timestamps, and drops pending output.
</Step>
<Step title="The transport flushes unplayed audio">
The output transport drains its audio queue, discarding audio that was
generated but not yet played. If a background audio mixer is active, the
transport drains only the bot's speech so the background audio keeps playing
without a gap.
</Step>
</Steps>

The result: the bot goes silent within roughly one audio write, and the pipeline is clean and ready for the user's new turn.

## What ends up in the context

A common question: if the bot is cut off mid-sentence, what does the LLM context contain?

**Only the words that were actually spoken.** As the bot speaks, the output transport pushes `TTSTextFrame`s downstream in sync with audio playback. Text that never played never reaches the assistant context aggregator. On interruption, the aggregator commits the partial, spoken-so-far text to the context as the assistant message.

This means the LLM's next completion sees an accurate transcript of the conversation: the bot's message ends where the user cut it off, not where the LLM's generation ended.

You can observe this with the `on_assistant_turn_stopped` event, which reports the committed text and whether the turn was interrupted:

```python
@assistant_aggregator.event_handler("on_assistant_turn_stopped")
async def on_assistant_turn_stopped(aggregator, message):
if message.interrupted:
print(f"Bot was cut off after saying: {message.content}")
```

## Controlling interruptions

**Let the bot finish speaking** with [user input muting](/pipecat/fundamentals/user-input-muting). Mute strategies block user audio, transcriptions, and interruption signals while active, so speech over the bot is discarded rather than answered later. Use `AlwaysUserMuteStrategy` to mute whenever the bot is speaking, or `FirstSpeechUserMuteStrategy` to protect just the introduction:

```python
from pipecat.processors.aggregators.llm_response_universal import (
LLMContextAggregatorPair,
LLMUserAggregatorParams,
)
from pipecat.turns.user_mute import AlwaysUserMuteStrategy

user_aggregator, assistant_aggregator = LLMContextAggregatorPair(
context,
user_params=LLMUserAggregatorParams(
user_mute_strategies=[AlwaysUserMuteStrategy()],
),
)
```

**Require a minimum number of words** so short utterances like "okay" or "yeah" don't interrupt the bot. This is configured on the [user turn start strategy](/api-reference/server/utilities/turn-management/user-turn-strategies#start-strategies):

```python
from pipecat.turns.user_start import MinWordsUserTurnStartStrategy

start_strategy = MinWordsUserTurnStartStrategy(min_words=3)
```

**Filter out backchannels with a model.** The [Krisp VIVA Interruption Prediction strategy](/api-reference/server/utilities/turn-management/user-turn-strategies#krispvivaipuserturnstartstrategy) distinguishes genuine interruptions from acknowledgments like "uh-huh".

**Disable interruptions** so in-flight work is never cancelled:

```python
from pipecat.turns.user_start import VADUserTurnStartStrategy

start_strategy = VADUserTurnStartStrategy(enable_interruptions=False)
```

<Warning>
Disabling interruptions does not ignore the user. Speech over the bot is
still transcribed and processed as a normal user turn — the bot's reply is
queued and plays as soon as the current speech finishes. If you want the bot
to finish speaking *and* discard what the user said over it, use mute
strategies instead.
</Warning>

## Triggering an interruption yourself

Sometimes the bot should stop itself: a timeout fires, an external event arrives, or your own logic decides the current response is no longer relevant.

From inside a custom `FrameProcessor`, broadcast the interruption directly:

```python
await self.broadcast_interruption()
```

From code that has a reference to a processor or the worker, you can also push an `InterruptionWorkerFrame`. The pipeline worker converts it into an `InterruptionFrame` and sends it through the whole pipeline:

```python
from pipecat.frames.frames import InterruptionWorkerFrame

await worker.queue_frame(InterruptionWorkerFrame())
```

Both approaches run the same interruption flow described above: the bot stops speaking, in-flight work is cancelled, and the spoken-so-far text is committed to context.

## Speech-to-speech services

Realtime speech-to-speech services (like Gemini Live and OpenAI Realtime) handle interruption detection on the provider side. In these pipelines, the service emits the speaking events and Pipecat's aggregators follow along using [external turn strategies](/api-reference/server/utilities/turn-management/user-turn-strategies#externaluserturnstartstrategy). The provider decides when the user barged in; Pipecat still flushes local audio output so the bot goes silent right away.

## Related

- [Speech Input & Turn Detection](/pipecat/learn/speech-input) - how user turns are detected
- [User Turn Strategies](/api-reference/server/utilities/turn-management/user-turn-strategies) - full strategy reference
- [User Input Muting](/pipecat/fundamentals/user-input-muting) - suppress user input while the bot speaks
- [System Frames](/api-reference/server/frames/system-frames) - `InterruptionFrame` reference
2 changes: 1 addition & 1 deletion pipecat/get-started/introduction.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: "Pipecat"
description: "Open source Python framework for building voice and multimodal AI agents."
---

Pipecat is an open source Python framework for building voice and multimodal AI agents. It orchestrates AI services, network transports, and audio processing to enable ultra-low latency conversations that feel natural and responsive.
Pipecat is an open source Python framework for building voice and multimodal AI agents. It orchestrates AI services, network transports, and audio processing to enable ultra-low latency conversations that feel natural and responsive. The framework is free to use under the BSD-2 license, and you choose your own stack: any speech-to-text, LLM, and text-to-speech provider, hosted on your own infrastructure or [Pipecat Cloud](https://www.daily.co/pricing/pipecat-cloud) — you pay only for the providers you choose.

<Card title="Quickstart" icon="rocket" href="/pipecat/get-started/quickstart">
Want to dive right in? Build and run your first Pipecat application
Expand Down
27 changes: 15 additions & 12 deletions pipecat/get-started/quickstart.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -225,11 +225,11 @@ tts = CartesiaTTSService(
voice=os.getenv("CARTESIA_VOICE_ID", "71a7ad14-091c-4e8e-a314-022ece01c121"),
),
)
llm = OpenAIRealtimeLLMService(
llm = OpenAIResponsesLLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAIRealtimeLLMService.Settings(
settings=OpenAIResponsesLLMService.Settings(
model=os.getenv("OPENAI_MODEL", "gpt-4.1"),
system_instruction="You are a helpful assistant.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)
```
Expand Down Expand Up @@ -297,25 +297,27 @@ The worker handles pipeline execution, collects metrics, and manages RTVI events
Event handlers manage the bot's lifecycle and user interactions:

```python
# Kick off the conversation when the client signals it's ready
@worker.rtvi.event_handler("on_client_ready")
async def on_client_ready(rtvi):
context.add_message({"role": "developer", "content": "Start by concisely introducing yourself."})
await worker.queue_frames([LLMRunFrame()])

# Event handler for when a client connects
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):
logger.info(f"Client connected")
# Add a greeting message to the context
context.add_message("developer", "Say hello and briefly introduce yourself.")
# Prompt the bot to start talking when the client connects
await worker.queue_frames([LLMRunFrame()])
logger.info("Client connected")

# Event handler for when a client disconnects
@transport.event_handler("on_client_disconnected")
async def on_client_disconnected(transport, client):
logger.info(f"Client disconnected")
logger.info("Client disconnected")
# Cancel the worker when the client disconnects
# This stops the pipeline and all processors, cleaning up resources
await worker.cancel()
```

When a client connects, the bot adds a greeting instruction and queues a context frame to initiate the conversation. When disconnecting, it properly cancels the worker to clean up resources.
When the client signals it's ready over the RTVI protocol, the bot adds a greeting instruction and queues an `LLMRunFrame` to start the conversation. When the client disconnects, the worker is cancelled to clean up resources.

### Running the Pipeline

Expand All @@ -325,9 +327,10 @@ Finally, the pipeline is executed by a WorkerRunner:
# Create a WorkerRunner to run the worker
runner = WorkerRunner(handle_sigint=False)

# Finally, run the worker using the runner
# Finally, add the worker and run it
# This will start the pipeline and begin processing frames
await runner.run(worker)
await runner.add_workers(worker)
await runner.run()
```

The runner manages the pipeline's execution lifecycle. Note that `handle_sigint=False` because the main runner handles system signals.
Expand Down
4 changes: 3 additions & 1 deletion pipecat/learn/context-management.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -215,7 +215,9 @@ Place the assistant context aggregator **after `transport.output()`**. This posi

<Tip>
Always place the assistant context aggregator **after** `transport.output()`
to ensure proper word-level context updates during interruptions.
to ensure proper word-level context updates during interruptions. See
[Interruptions](/pipecat/fundamentals/interruptions#what-ends-up-in-the-context)
for what the context contains when the bot is cut off mid-sentence.
</Tip>

## Manual Context Control
Expand Down
21 changes: 2 additions & 19 deletions pipecat/learn/speech-input.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -156,26 +156,9 @@ stop_strategy = SpeechTimeoutUserTurnStopStrategy(user_speech_timeout=0.6)

### Interruptions

Interruptions stop the bot when the user starts speaking. This is controlled by the `enable_interruptions` parameter on start strategies (enabled by default).
When a user turn starts with interruptions enabled (the default, via the `enable_interruptions` parameter on start strategies), the bot immediately stops speaking, in-flight work is cancelled, and the pipeline is ready for the new user input.

When a user turn starts with interruptions enabled:

1. Bot immediately stops speaking
2. Pending audio and text is cleared
3. Pipeline ready for new user input

To disable interruptions:

```python
from pipecat.turns.user_start import VADUserTurnStartStrategy

start_strategy = VADUserTurnStartStrategy(enable_interruptions=False)
```

<Note>
Keep interruptions enabled (default) for natural conversations. This enables
users to interrupt the bot mid-sentence, just like human conversations.
</Note>
Keep interruptions enabled for natural conversations. For how the cancellation works, what ends up in the LLM context, and how to tune, disable, or trigger interruptions yourself, see [Interruptions](/pipecat/fundamentals/interruptions).

## Best Practices

Expand Down
Loading