Skip to content

RHIDP-15908: [Lightspeed] Remove Llama Stack API and config dependencies #2544

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jmagak wants to merge 1 commit into from
Closed

RHIDP-15908: [Lightspeed] Remove Llama Stack API and config dependencies #2544

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 4 additions & 9 deletions modules/shared/con-ai-reference-and-tool-calling-capabilities-through.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,29 +4,24 @@
= AI reference and tool-calling capabilities through {lcs-name}

[role="_abstract"]
Review the core components managed by the {lcs-name}({lcs-short}) sidecar container to plan integrations with large language models (LLM) and tool runtime providers.
You can configure AI integrations through the {lcs-name} ({lcs-short}) sidecar container, which manages all connections between {product-very-short} and your LLM and tool runtime providers.

The {lcs-short} container deploys as a sidecar to extend {product-very-short} functionality. The container integrates and manages the following core architectural components:
{lcs-short} integrates and manages the following core components:

* Large language model (LLM) inference providers

* Model Context Protocol (MCP) or Retrieval Augmented Generation (RAG) tool runtime providers
+
[IMPORTANT]
====
Verify that your model supports tool calling before you enable MCP features. Using an incompatible model results in error messages.
====

* Safety providers

* Vector database settings

{lcs-short} also manages critical operational configuration and key data, specifically:
{lcs-short} also manages the following operational data:

* User feedback collection

* MCP server configuration

* Chat history

{ls-short} sends prompts and receives LLM responses through the {lcs-short} sidecar.
All prompts and LLM responses pass through the {lcs-short} sidecar.
6 changes: 4 additions & 2 deletions modules/shared/con-architecture-for-your-ai-backend-deployment.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,11 @@
= {ls-brand-name} architecture for your AI backend deployment

[role="_abstract"]
Review the {ls-short} component architecture to plan your system layout and coordinate connections with your artificial intelligence (AI) backend deployment.
You can plan your AI backend deployment by understanding how {ls-short} uses the {lcs-name} ({lcs-short}) container to connect {product-very-short} with your large language model (LLM) providers.

The architecture relies on the {lcs-name} ({lcs-short}) container, which operates as the primary intermediary layer to manage {ls-short} functionality and console user interactions. By default, the interface appears as a floating action button (FAB) on all platforms that host {product-very-short}.
The {lcs-short} container serves as the sole integration layer between {product-very-short} and your AI backend services. All LLM inference, tool calling, safety, and retrieval-augmented generation (RAG) requests pass through {lcs-short}, which runs as a sidecar container alongside {product-very-short}.

By default, {product-very-short} displays the {ls-short} interface as a floating action button (FAB) on all platforms.

.Additional resources
* link:https://access.redhat.com/support/policy/updates/developerhub[{product} Life Cycle and supported platforms]
Loading