diff --git a/api-reference/server/services/llm/crusoe.mdx b/api-reference/server/services/llm/crusoe.mdx new file mode 100644 index 00000000..3e0b6ba2 --- /dev/null +++ b/api-reference/server/services/llm/crusoe.mdx @@ -0,0 +1,162 @@ +--- +title: "Crusoe" +description: "Large Language Model services using Crusoe Cloud's Managed Inference OpenAI-compatible API" +--- + +## Overview + +`CrusoeLLMService` provides chat completion capabilities using Crusoe Cloud's Managed Inference API with OpenAI-compatible interface. It supports streaming responses and function calling. + + + + Pipecat's API methods for Crusoe integration + + + Function calling example with Crusoe + + + Official Crusoe documentation + + + Access models and manage API keys + + + +## Installation + +To use Crusoe LLM services, install the required dependencies: + +```bash +uv add "pipecat-ai[crusoe]" +``` + +## Prerequisites + +### Crusoe Account Setup + +Before using Crusoe LLM services, you need: + +1. **Crusoe Account**: Sign up at [Crusoe Cloud](https://crusoe.ai/) +2. **API Key**: Generate an API key from your account dashboard +3. **Model Selection**: Choose from available models (default: `zai/GLM-5.2`) + +### Required Environment Variables + +- `CRUSOE_API_KEY`: Your Crusoe API key for authentication + +## Configuration + + + The API key for accessing Crusoe's Managed Inference API. + + + + The base URL for the Crusoe API. Override if using a different endpoint. + + + + Runtime-configurable model settings. See [Settings](#settings) below. + + +### Settings + +Runtime-configurable settings passed via the `settings` constructor argument using `CrusoeLLMService.Settings(...)`. These can be updated mid-conversation with `LLMUpdateSettingsFrame`. See [Service Settings](/pipecat/fundamentals/service-settings) for details. + +| Parameter | Type | Default | Description | +| ------------------- | ------- | -------------- | -------------------------------------------------------------------------------------- | +| `model` | `str` | `"zai/GLM-5.2"` | Crusoe model identifier. Check Crusoe Cloud for available models. | +| `temperature` | `float` | `NOT_GIVEN` | Sampling temperature (0.0 to 2.0). Lower values are more focused, higher are creative. | +| `max_tokens` | `int` | `NOT_GIVEN` | Maximum tokens to generate. | +| `top_p` | `float` | `NOT_GIVEN` | Top-p (nucleus) sampling (0.0 to 1.0). Controls diversity of output. | +| `frequency_penalty` | `float` | `NOT_GIVEN` | Penalty for frequent tokens (-2.0 to 2.0). Positive values discourage repetition. | +| `presence_penalty` | `float` | `NOT_GIVEN` | Penalty for new topics (-2.0 to 2.0). Positive values encourage new topics. | + + + `NOT_GIVEN` values are omitted from the API request entirely, letting the + Crusoe API use its own defaults. This is different from `None`, which would be + sent explicitly. + + +## Usage + +### Basic Setup + +```python +import os +from pipecat.services.crusoe import CrusoeLLMService + +llm = CrusoeLLMService( + api_key=os.getenv("CRUSOE_API_KEY"), +) +``` + +### With Custom Settings + +```python +import os +from pipecat.services.crusoe import CrusoeLLMService + +llm = CrusoeLLMService( + api_key=os.getenv("CRUSOE_API_KEY"), + settings=CrusoeLLMService.Settings( + model="zai/GLM-5.2", + temperature=0.7, + max_tokens=1000, + ), +) +``` + +### Updating Settings at Runtime + +Model settings can be changed mid-conversation using `LLMUpdateSettingsFrame`: + +```python +from pipecat.frames.frames import LLMUpdateSettingsFrame +from pipecat.services.crusoe.llm import CrusoeLLMSettings + +await worker.queue_frame( + LLMUpdateSettingsFrame( + delta=CrusoeLLMSettings( + temperature=0.3, + ) + ) +) +``` + +## Notes + +- **OpenAI Compatibility**: Crusoe's API is OpenAI-compatible, allowing use of familiar patterns and parameters. +- **Function Calling**: Supports OpenAI-style tool/function calling format. +- **Streaming**: Supports streaming responses for real-time interaction. +- **Developer Role**: Crusoe does not support the `developer` message role. Messages with this role will be automatically converted to `system` messages. + +## Event Handlers + +`CrusoeLLMService` supports the following event handlers, inherited from [LLMService](/server/events/service-events): + +| Event | Description | +| --------------------------- | ----------------------------------------------------------------------- | +| `on_completion_timeout` | Called when an LLM completion request times out | +| `on_function_calls_started` | Called when function calls are received and execution is about to start | + +```python +@llm.event_handler("on_completion_timeout") +async def on_completion_timeout(service): + print("LLM completion timed out") +``` diff --git a/api-reference/server/services/supported-services.mdx b/api-reference/server/services/supported-services.mdx index d14c9230..f922c479 100644 --- a/api-reference/server/services/supported-services.mdx +++ b/api-reference/server/services/supported-services.mdx @@ -65,6 +65,7 @@ LLMs receive text or audio based input and output a streaming text response. | [AWS Bedrock](/api-reference/server/services/llm/aws) | `uv add "pipecat-ai[aws]"` | Pipecat | | [Azure](/api-reference/server/services/llm/azure) | `uv add "pipecat-ai[azure]"` | Pipecat | | [Cerebras](/api-reference/server/services/llm/cerebras) | `uv add "pipecat-ai[cerebras]"` | Pipecat | +| [Crusoe](/api-reference/server/services/llm/crusoe) | `uv add "pipecat-ai[crusoe]"` | Pipecat | | [DeepSeek](/api-reference/server/services/llm/deepseek) | `uv add "pipecat-ai[deepseek]"` | Pipecat | | [Fireworks AI](/api-reference/server/services/llm/fireworks) | `uv add "pipecat-ai[fireworks]"` | Pipecat | | [Google Gemini](/api-reference/server/services/llm/google) | `uv add "pipecat-ai[google]"` | Pipecat | diff --git a/docs.json b/docs.json index 5eb257e1..7df22bcb 100644 --- a/docs.json +++ b/docs.json @@ -390,6 +390,7 @@ "api-reference/server/services/llm/aws", "api-reference/server/services/llm/azure", "api-reference/server/services/llm/cerebras", + "api-reference/server/services/llm/crusoe", "api-reference/server/services/llm/deepseek", "api-reference/server/services/llm/fireworks", "api-reference/server/services/llm/google",