Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions examples/list/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ This example demonstrates how to use the Inference Gateway Python SDK to list av
- Connect to the Inference Gateway
- List all available models
- Filter models by provider (e.g., OpenAI, Groq, etc.)
- Include additional model metadata (context_window, pricing, modalities)

Run the example with:

Expand Down
4 changes: 4 additions & 0 deletions examples/list/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,7 @@
# Filter by provider
openai_models = client.list_models(provider=PROVIDER)
print(f"Provider {PROVIDER} models:", openai_models)

# Include additional metadata (context_window, pricing, modalities)
detailed_models = client.list_models(include=["context_window", "pricing", "modalities"])
print("Models with details:", detailed_models)
2 changes: 1 addition & 1 deletion inference_gateway/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,7 @@ def list_models(
Args:
provider: Optional provider to filter models
include: Optional list of additional metadata to include.
Supported values: "context_window", "pricing".
Supported values: "context_window", "pricing", "modalities".

Returns:
ListModelsResponse: List of available models
Expand Down
4 changes: 4 additions & 0 deletions inference_gateway/models.py

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

55 changes: 37 additions & 18 deletions openapi.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -86,10 +86,11 @@ paths:
type: string
enum:
- context_window
- modalities
- pricing
description: |
Comma-separated list of metadata keys to include in the response.
Supported values: `pricing`, `context_window`.
Supported values: `pricing`, `context_window`, `modalities`.
When omitted, the response remains unchanged (backward compatible).
responses:
'200':
Expand Down Expand Up @@ -244,14 +245,35 @@ paths:
context_window:
tokens: 128000
source: 'provider'
includeModalities:
summary: Models with modalities metadata
value:
object: 'list'
data:
- id: 'openai/gpt-4o'
object: 'model'
created: 1686935002
owned_by: 'openai'
served_by: 'openai'
modalities:
- text
- image
- id: 'openai/gpt-4-turbo'
object: 'model'
created: 1687882410
owned_by: 'openai'
served_by: 'openai'
modalities:
- text
- image
'400':
description: Bad request - unsupported include value
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
example:
error: "Unsupported include value: 'unsupported'. Supported values: pricing, context_window"
error: "Unsupported include value: 'unsupported'. Supported values: pricing, context_window, modalities"
'401':
$ref: '#/components/responses/Unauthorized'
'500':
Expand Down Expand Up @@ -1042,7 +1064,6 @@ components:
id: 'ollama'
url: 'http://ollama:8080/v1'
auth_type: 'none'
supports_vision: true
endpoints:
models:
name: 'list_models'
Expand All @@ -1056,7 +1077,6 @@ components:
id: 'ollama_cloud'
url: 'https://ollama.com/v1'
auth_type: 'bearer'
supports_vision: true
endpoints:
models:
name: 'list_models'
Expand All @@ -1070,7 +1090,6 @@ components:
id: 'anthropic'
url: 'https://api.anthropic.com/v1'
auth_type: 'xheader'
supports_vision: true
extra_headers:
anthropic-version: '2023-06-01'
endpoints:
Expand All @@ -1086,7 +1105,6 @@ components:
id: 'cohere'
url: 'https://api.cohere.ai'
auth_type: 'bearer'
supports_vision: true
endpoints:
models:
name: 'list_models'
Expand All @@ -1100,7 +1118,6 @@ components:
id: 'groq'
url: 'https://api.groq.com/openai/v1'
auth_type: 'bearer'
supports_vision: true
endpoints:
models:
name: 'list_models'
Expand All @@ -1114,7 +1131,6 @@ components:
id: 'llamacpp'
url: 'http://llamacpp:8080/v1'
auth_type: 'bearer'
supports_vision: true
endpoints:
models:
name: 'list_models'
Expand All @@ -1128,7 +1144,6 @@ components:
id: 'openai'
url: 'https://api.openai.com/v1'
auth_type: 'bearer'
supports_vision: true
endpoints:
models:
name: 'list_models'
Expand Down Expand Up @@ -1158,7 +1173,6 @@ components:
id: 'cloudflare'
url: 'https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai'
auth_type: 'bearer'
supports_vision: false
endpoints:
models:
name: 'list_models'
Expand All @@ -1172,7 +1186,6 @@ components:
id: 'deepseek'
url: 'https://api.deepseek.com'
auth_type: 'bearer'
supports_vision: false
endpoints:
models:
name: 'list_models'
Expand All @@ -1186,7 +1199,6 @@ components:
id: 'google'
url: 'https://generativelanguage.googleapis.com/v1beta/openai'
auth_type: 'bearer'
supports_vision: true
endpoints:
models:
name: 'list_models'
Expand All @@ -1200,7 +1212,6 @@ components:
id: 'mistral'
url: 'https://api.mistral.ai/v1'
auth_type: 'bearer'
supports_vision: true
endpoints:
models:
name: 'list_models'
Expand All @@ -1214,7 +1225,6 @@ components:
id: 'minimax'
url: 'https://api.minimax.io/v1'
auth_type: 'bearer'
supports_vision: true
endpoints:
models:
name: 'list_models'
Expand All @@ -1228,7 +1238,6 @@ components:
id: 'moonshot'
url: 'https://api.moonshot.ai/v1'
auth_type: 'bearer'
supports_vision: true
endpoints:
models:
name: 'list_models'
Expand All @@ -1242,7 +1251,6 @@ components:
id: 'nvidia'
url: 'https://integrate.api.nvidia.com/v1'
auth_type: 'bearer'
supports_vision: true
endpoints:
models:
name: 'list_models'
Expand All @@ -1256,7 +1264,6 @@ components:
id: 'zai'
url: 'https://api.z.ai/api/paas/v4'
auth_type: 'bearer'
supports_vision: true
endpoints:
models:
name: 'list_models'
Expand Down Expand Up @@ -1621,6 +1628,18 @@ components:
type: string
served_by:
$ref: '#/components/schemas/Provider'
modalities:
oneOf:
- type: array
items:
type: string
enum:
- text
- image
- audio
- video
- type: 'null'
description: The modalities the model supports natively (included when `include=modalities`)
context_window:
oneOf:
- $ref: '#/components/schemas/ContextWindow'
Expand Down Expand Up @@ -3988,7 +4007,7 @@ components:
- name: host
env: 'SERVER_HOST'
type: string
default: '0.0.0.0'
default: '127.0.0.1'
description: 'Server host'
- name: port
env: 'SERVER_PORT'
Expand Down