Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -239,6 +239,10 @@ jobs:
run: npx tsx anthropic_client.ts
working-directory: examples/node

- name: Run TypeScript image example
run: npx tsx images_client.ts
working-directory: examples/node

- name: Run Go Anthropic example
run: go run anthropic_client.go
working-directory: examples/go
Expand All @@ -260,7 +264,8 @@ jobs:
openresponses_client langchain_client \
langchain_responses_client openai_websocket_client \
pydantic_ai_client \
anthropic_client anthropic_langchain)
anthropic_client anthropic_langchain \
images_client)
declare -A pid
for ex in "${examples[@]}"; do
uv run "examples/$ex.py" > "$RUNNER_TEMP/$ex.log" 2>&1 &
Expand Down
6 changes: 5 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ Available specs:
- `specs/api-endpoints.md` - API endpoint structure and routing conventions
- `specs/responses-api.md` - OpenAI Responses API simulation
- `specs/anthropic-api.md` - Anthropic Messages API simulation
- `specs/image-generation.md` - OpenAI image generation (gpt-image / "ChatGPT Images") simulation
- `specs/scripted-mode.md` - Multi-turn scripted responses for agent scenario tests
- `specs/load-testing.md` - Load testing framework and benchmarks
- `specs/throughput-benchmark.md` - Peak throughput (req/s) benchmark with parallelisation scaling
Expand Down Expand Up @@ -99,6 +100,7 @@ Examples:
**Current OpenAI endpoints:**
- `POST /openai/v1/chat/completions` - Chat completions (streaming supported)
- `POST /openai/v1/responses` - Responses API (streaming supported)
- `POST /openai/v1/images/generations` - Image generation (gpt-image, streaming supported)
- `GET /openai/v1/models` - List available models
- `GET /openai/v1/models/:id` - Get model details

Expand All @@ -119,7 +121,9 @@ See `specs/api-endpoints.md` for the full specification.
#### Code organization

- `src/` - Main source code (library and binary)
- `src/openai/` - OpenAI API types and model profiles
- `src/openai/` - OpenAI API types and model profiles (includes `images.rs` for image generation)
- `src/imagegen.rs` - Self-contained placeholder PNG synthesis (encoder + bitmap font)
- `src/image_stream.rs` - Image generation streaming engine (partial images)
- `src/openresponses/` - OpenResponses API types and streaming
- `src/anthropic/` - Anthropic Messages API types, model profiles, and streaming
- `src/cli/` - CLI server, configuration, and HTTP handlers
Expand Down
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ LLMSim replicates realistic LLM API behavior without running actual models. It s
- **Multi-Provider API Support** - OpenAI Chat Completions, [OpenResponses](https://www.openresponses.org), and Anthropic Messages APIs
- **Realistic Latency Simulation** - Time-to-first-token (TTFT) and inter-token delays with normal distribution
- **Streaming Support** - Server-Sent Events (SSE) for OpenAI, OpenResponses, and Anthropic streaming formats
- **Image Generation** - Simulated gpt-image ("ChatGPT Images") endpoint returning watermarked PNGs, with streaming partial images
- **Accurate Token Counting** - Uses tiktoken-rs (OpenAI's tokenizer implementation)
- **Error Injection** - Rate limits (429), server errors (500/503), timeouts
- **Multiple Response Generators** - Lorem ipsum, echo, fixed, random, sequence
Expand Down Expand Up @@ -131,6 +132,7 @@ Note that `count_tokens` and the `tokens` module are only available with the
| `/openai/v1/models` | GET | List available models |
| `/openai/v1/models/{model_id}` | GET | Get specific model details |
| `/openai/v1/responses` | POST | Responses API (streaming & non-streaming) |
| `/openai/v1/images/generations` | POST | Image generation (gpt-image, streaming & non-streaming) |

When using OpenAI SDKs, set the base URL to `http://localhost:8080/openai/v1`.

Expand Down
76 changes: 76 additions & 0 deletions docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ LLMSim provides multiple API providers with provider-namespaced routes.
| `/openai/v1/chat/completions` | POST | Chat completions (streaming & non-streaming) |
| `/openai/v1/responses` | POST | Responses API (streaming & non-streaming) |
| `/openai/v1/responses` | WS | WebSocket mode for Responses API |
| `/openai/v1/images/generations` | POST | Image generation (streaming & non-streaming) |
| `/openai/v1/models` | GET | List available models |
| `/openai/v1/models/:id` | GET | Get model details |

Expand Down Expand Up @@ -214,6 +215,81 @@ curl http://localhost:8080/openai/v1/models
curl http://localhost:8080/openai/v1/models/gpt-5
```

### Image Generation

Simulates the gpt-image family ("ChatGPT Images"). Returns a synthetic PNG of
the requested size that renders the prompt text and a "LLMSIM SIMULATED IMAGE"
watermark — no real model runs. Works with the official SDKs
(`client.images.generate(...)`).

```bash
curl http://localhost:8080/openai/v1/images/generations \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-image-1",
"prompt": "a cat riding a bicycle",
"size": "1024x1024",
"quality": "low"
}'
```

#### Request Fields

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `prompt` | string | (required) | Text description of the image |
| `model` | string | `gpt-image-1` | `gpt-image-1`, `gpt-image-1-mini`, `gpt-image-1.5` |
| `n` | integer | `1` | Number of images (1–10) |
| `size` | string | `1024x1024` | `1024x1024`, `1536x1024`, `1024x1536`, or `auto` |
| `quality` | string | `high` | `low`, `medium`, `high`, `auto` |
| `output_format` | string | `png` | `png`, `jpeg`, `webp` (echoed back) |
| `background` | string | `opaque` | `transparent`, `opaque`, `auto` |
| `stream` | boolean | `false` | Stream partial images via SSE |
| `partial_images` | integer | `0` | Progressive preview frames (0–3) |

#### Response

```json
{
"created": 1750000000,
"data": [{ "b64_json": "<base64 PNG>" }],
"usage": {
"input_tokens": 8,
"output_tokens": 272,
"total_tokens": 280,
"input_tokens_details": { "text_tokens": 8, "image_tokens": 0 }
},
"size": "1024x1024",
"quality": "low",
"output_format": "png",
"background": "opaque"
}
```

#### Streaming

With `"stream": true` and `"partial_images": N`, the server emits SSE events:
`image_generation.partial_image` frames (progressively sharper previews) followed
by a final `image_generation.completed` event carrying the full image and usage.

```bash
curl http://localhost:8080/openai/v1/images/generations \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-image-1",
"prompt": "sunset over mountains",
"stream": true,
"partial_images": 3
}'
```

Generation latency scales with quality and image size and is anchored to the
configured latency profile; the `instant`/`fast` profiles make it effectively
immediate for tests and load runs.

See [`specs/image-generation.md`](../specs/image-generation.md) for the full
specification.

## OpenResponses API (`/openresponses/v1/...`)

[OpenResponses](https://www.openresponses.org) is an open-source specification for building multi-provider, interoperable LLM interfaces.
Expand Down
32 changes: 32 additions & 0 deletions examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,16 @@ Using the OpenResponses specification with httpx:
uv run examples/openresponses_client.py
```

### Image Generation

Generate simulated images (gpt-image / "ChatGPT Images"), including streaming
with progressive partial images. Each image is a synthetic PNG of the requested
size rendering the prompt text and a "LLMSIM SIMULATED IMAGE" watermark:

```bash
uv run examples/images_client.py
```

### WebSocket Mode

Using the WebSocket transport for the Responses API:
Expand Down Expand Up @@ -151,6 +161,15 @@ Direct usage of the official OpenAI Node.js library:
npx tsx openai_client.ts
```

### OpenAI Image Generation

Generate simulated images (gpt-image / "ChatGPT Images"), including streaming
with progressive partial images:

```bash
npx tsx images_client.ts
```

### Vercel AI SDK

Using the [Vercel AI SDK](https://sdk.vercel.ai) via its OpenAI-compatible provider:
Expand Down Expand Up @@ -242,6 +261,19 @@ curl http://localhost:8080/openai/v1/responses \
}'
```

### OpenAI Image Generation

```bash
curl http://localhost:8080/openai/v1/images/generations \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-image-1",
"prompt": "a cat riding a bicycle",
"size": "1024x1024",
"quality": "low"
}'
```

### OpenResponses API

```bash
Expand Down
150 changes: 150 additions & 0 deletions examples/images_client.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.10"
# dependencies = [
# "openai>=2.44.0",
# "httpx>=0.27.0",
# ]
# ///
"""
Image generation example for llmsim (gpt-image / "ChatGPT Images").

This script demonstrates the simulated OpenAI image generation endpoint. The
server returns a synthetic PNG of the requested size that renders the prompt
text and a clear "LLMSIM SIMULATED IMAGE" watermark — no real model runs.

Server endpoint:
POST /openai/v1/images/generations - Image generation (streaming supported)

Prerequisites:
Start the llmsim server first:
llmsim serve --port 8080

Or from source:
cargo run --release -- serve --port 8080

Usage:
uv run examples/images_client.py

Environment variables:
LLMSIM_URL: Server URL (default: http://localhost:8080/openai/v1)
"""

import base64
import json
import os
import sys
import tempfile

import httpx
from openai import OpenAI


def main() -> None:
base_url = os.environ.get("LLMSIM_URL", "http://localhost:8080/openai/v1")
out_dir = tempfile.mkdtemp(prefix="llmsim-images-")

print("=" * 50)
print("OpenAI Image Generation + LLMSim Example")
print("=" * 50)
print(f"\nConnecting to: {base_url}")
print(f"Saving images to: {out_dir}")
print()

client = OpenAI(base_url=base_url, api_key="not-needed")

# Example 1: Basic image generation (non-streaming).
print("1. Generate an image")
print("-" * 30)
try:
result = client.images.generate(
model="gpt-image-1",
prompt="a cat riding a bicycle on the moon",
size="1024x1024",
quality="low",
)
except Exception as e: # noqa: BLE001
print(f"Error: {e}")
print("\nMake sure the llmsim server is running:")
print(" llmsim serve --port 8080")
sys.exit(1)

image = result.data[0]
png = base64.b64decode(image.b64_json)
path = os.path.join(out_dir, "moon_cat.png")
with open(path, "wb") as f:
f.write(png)
print(f"Saved {len(png):,} bytes -> {path}")
print(f"Usage: {result.usage}")
print()

# Example 2: Different sizes and qualities.
print("2. Sizes and qualities")
print("-" * 30)
for size, quality in [("1024x1024", "low"), ("1536x1024", "medium"), ("1024x1536", "high")]:
result = client.images.generate(
model="gpt-image-1",
prompt=f"a {quality} quality landscape, {size}",
size=size,
quality=quality,
)
png = base64.b64decode(result.data[0].b64_json)
print(f" {size} {quality:>6}: {len(png):,} bytes, output_tokens={result.usage.output_tokens}")
print()

# Example 3: Streaming with partial images.
#
# The OpenAI image streaming API emits `image_generation.partial_image`
# events (progressively sharper previews) followed by a final
# `image_generation.completed` event carrying the full image and usage.
# We parse the SSE stream directly with httpx so the event shape is explicit.
print("3. Streaming with partial images")
print("-" * 30)
with httpx.Client(base_url=base_url, timeout=120.0) as http:
with http.stream(
"POST",
"/images/generations",
json={
"model": "gpt-image-1",
"prompt": "sunset over snowy mountains",
"size": "1024x1024",
"quality": "medium",
"stream": True,
"partial_images": 3,
},
) as resp:
buf = ""
for chunk in resp.iter_text():
buf += chunk
while "\n\n" in buf:
raw, buf = buf.split("\n\n", 1)
data_line = next(
(ln[6:] for ln in raw.splitlines() if ln.startswith("data: ")),
None,
)
if not data_line:
continue
event = json.loads(data_line)
etype = event["type"]
png = base64.b64decode(event["b64_json"])
if etype == "image_generation.partial_image":
idx = event["partial_image_index"]
path = os.path.join(out_dir, f"stream_partial_{idx}.png")
with open(path, "wb") as f:
f.write(png)
print(f" partial #{idx}: {len(png):,} bytes -> {path}")
elif etype == "image_generation.completed":
path = os.path.join(out_dir, "stream_final.png")
with open(path, "wb") as f:
f.write(png)
print(f" completed : {len(png):,} bytes -> {path}")
print(f" usage : {event['usage']}")
print()

print("=" * 50)
print("Examples complete!")
print("=" * 50)


if __name__ == "__main__":
main()
Loading
Loading