Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
162 changes: 162 additions & 0 deletions api-reference/server/services/llm/crusoe.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
---
title: "Crusoe"
description: "Large Language Model services using Crusoe Cloud's Managed Inference OpenAI-compatible API"
---

## Overview

`CrusoeLLMService` provides chat completion capabilities using Crusoe Cloud's Managed Inference API with OpenAI-compatible interface. It supports streaming responses and function calling.

<CardGroup cols={2}>
<Card
title="Crusoe LLM API Reference"
icon="code"
href="https://reference-server.pipecat.ai/en/latest/api/pipecat.services.crusoe.llm.html"
>
Pipecat's API methods for Crusoe integration
</Card>
<Card
title="Example Implementation"
icon="play"
href="https://github.com/pipecat-ai/pipecat/blob/main/examples/function-calling/function-calling-crusoe.py"
>
Function calling example with Crusoe
</Card>
<Card title="Crusoe Documentation" icon="book" href="https://crusoe.ai/">
Official Crusoe documentation
</Card>
<Card
title="Crusoe Cloud"
icon="globe"
href="https://crusoe.ai/"
>
Access models and manage API keys
</Card>
</CardGroup>

## Installation

To use Crusoe LLM services, install the required dependencies:

```bash
uv add "pipecat-ai[crusoe]"
```

## Prerequisites

### Crusoe Account Setup

Before using Crusoe LLM services, you need:

1. **Crusoe Account**: Sign up at [Crusoe Cloud](https://crusoe.ai/)
2. **API Key**: Generate an API key from your account dashboard
3. **Model Selection**: Choose from available models (default: `zai/GLM-5.2`)

### Required Environment Variables

- `CRUSOE_API_KEY`: Your Crusoe API key for authentication

## Configuration

<ParamField path="api_key" type="str" required>
The API key for accessing Crusoe's Managed Inference API.
</ParamField>

<ParamField
path="base_url"
type="str"
default="https://api.inference.crusoecloud.com/v1/"
>
The base URL for the Crusoe API. Override if using a different endpoint.
</ParamField>

<ParamField path="settings" type="CrusoeLLMService.Settings" default="None">
Runtime-configurable model settings. See [Settings](#settings) below.
</ParamField>

### Settings

Runtime-configurable settings passed via the `settings` constructor argument using `CrusoeLLMService.Settings(...)`. These can be updated mid-conversation with `LLMUpdateSettingsFrame`. See [Service Settings](/pipecat/fundamentals/service-settings) for details.

| Parameter | Type | Default | Description |
| ------------------- | ------- | -------------- | -------------------------------------------------------------------------------------- |
| `model` | `str` | `"zai/GLM-5.2"` | Crusoe model identifier. Check Crusoe Cloud for available models. |
| `temperature` | `float` | `NOT_GIVEN` | Sampling temperature (0.0 to 2.0). Lower values are more focused, higher are creative. |
| `max_tokens` | `int` | `NOT_GIVEN` | Maximum tokens to generate. |
| `top_p` | `float` | `NOT_GIVEN` | Top-p (nucleus) sampling (0.0 to 1.0). Controls diversity of output. |
| `frequency_penalty` | `float` | `NOT_GIVEN` | Penalty for frequent tokens (-2.0 to 2.0). Positive values discourage repetition. |
| `presence_penalty` | `float` | `NOT_GIVEN` | Penalty for new topics (-2.0 to 2.0). Positive values encourage new topics. |

<Note>
`NOT_GIVEN` values are omitted from the API request entirely, letting the
Crusoe API use its own defaults. This is different from `None`, which would be
sent explicitly.
</Note>

## Usage

### Basic Setup

```python
import os
from pipecat.services.crusoe import CrusoeLLMService

llm = CrusoeLLMService(
api_key=os.getenv("CRUSOE_API_KEY"),
)
```

### With Custom Settings

```python
import os
from pipecat.services.crusoe import CrusoeLLMService

llm = CrusoeLLMService(
api_key=os.getenv("CRUSOE_API_KEY"),
settings=CrusoeLLMService.Settings(
model="zai/GLM-5.2",
temperature=0.7,
max_tokens=1000,
),
)
```

### Updating Settings at Runtime

Model settings can be changed mid-conversation using `LLMUpdateSettingsFrame`:

```python
from pipecat.frames.frames import LLMUpdateSettingsFrame
from pipecat.services.crusoe.llm import CrusoeLLMSettings

await worker.queue_frame(
LLMUpdateSettingsFrame(
delta=CrusoeLLMSettings(
temperature=0.3,
)
)
)
```

## Notes

- **OpenAI Compatibility**: Crusoe's API is OpenAI-compatible, allowing use of familiar patterns and parameters.
- **Function Calling**: Supports OpenAI-style tool/function calling format.
- **Streaming**: Supports streaming responses for real-time interaction.
- **Developer Role**: Crusoe does not support the `developer` message role. Messages with this role will be automatically converted to `system` messages.

## Event Handlers

`CrusoeLLMService` supports the following event handlers, inherited from [LLMService](/server/events/service-events):

| Event | Description |
| --------------------------- | ----------------------------------------------------------------------- |
| `on_completion_timeout` | Called when an LLM completion request times out |
| `on_function_calls_started` | Called when function calls are received and execution is about to start |

```python
@llm.event_handler("on_completion_timeout")
async def on_completion_timeout(service):
print("LLM completion timed out")
```
1 change: 1 addition & 0 deletions api-reference/server/services/supported-services.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ LLMs receive text or audio based input and output a streaming text response.
| [AWS Bedrock](/api-reference/server/services/llm/aws) | `uv add "pipecat-ai[aws]"` | Pipecat |
| [Azure](/api-reference/server/services/llm/azure) | `uv add "pipecat-ai[azure]"` | Pipecat |
| [Cerebras](/api-reference/server/services/llm/cerebras) | `uv add "pipecat-ai[cerebras]"` | Pipecat |
| [Crusoe](/api-reference/server/services/llm/crusoe) | `uv add "pipecat-ai[crusoe]"` | Pipecat |
| [DeepSeek](/api-reference/server/services/llm/deepseek) | `uv add "pipecat-ai[deepseek]"` | Pipecat |
| [Fireworks AI](/api-reference/server/services/llm/fireworks) | `uv add "pipecat-ai[fireworks]"` | Pipecat |
| [Google Gemini](/api-reference/server/services/llm/google) | `uv add "pipecat-ai[google]"` | Pipecat |
Expand Down
1 change: 1 addition & 0 deletions docs.json
Original file line number Diff line number Diff line change
Expand Up @@ -390,6 +390,7 @@
"api-reference/server/services/llm/aws",
"api-reference/server/services/llm/azure",
"api-reference/server/services/llm/cerebras",
"api-reference/server/services/llm/crusoe",
"api-reference/server/services/llm/deepseek",
"api-reference/server/services/llm/fireworks",
"api-reference/server/services/llm/google",
Expand Down
Loading