diff --git a/README.md b/README.md index f587c396..cb701148 100644 --- a/README.md +++ b/README.md @@ -108,7 +108,7 @@ main(); For comprehensive usage, development, and project state [read the docs](./docs/README.md). -- **Architecture**: Learn about our [library synchronization concept and data sources](./docs/architecture.md#data-sources-and-integrations). +- **Architecture**: Learn about our [architecture, collections, and data sources](./docs/architecture.md#collections-and-data-sources). - **Usage**: Detailed guide on [built-in tools, resources, and troubleshooting for general use](./docs/usage.md). - **Development**: Reference for [CLI options and tool plugins](./docs/development.md). diff --git a/docs/architecture.md b/docs/architecture.md index eeb49899..183a355e 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -12,8 +12,6 @@ The PatternFly MCP server is centered on a **Library of Records and Collections* - **Reading records**: Accessing full documentation and machine-readable schemas via exact hashes. - **Discovering collections**: Navigating the library via logical groupings of records. -> [A more in-depth version of our **library synchronization** concept is currently in progress](#library-synchronization-in-progress). - #### Discovery layer (library metadata) Instead of a standalone "discovery" tool, the server implements a robust **Library Metadata system**. This system: @@ -24,31 +22,28 @@ Instead of a standalone "discovery" tool, the server implements a robust **Libra > This discovery layer treats the MCP server as a living library. It enables the server to provide updates for all built-in tools and resources while maintaining a tailored experience based on user patterns (e.g., tailoring responses for designers vs. developers). -#### Collections and extensible sources +### Collections and data sources -The server organizes records across primary collections (such as `patternfly-docs`, `patternfly-component-schemas`, and `patternfly-api`). Outside of these core collections, additional specialized, supplemental, or experimental collections may be dynamically registered or evolve across server releases. +First and foremost, the MCP server is an application. Collections are aggregated to provide domain-specific knowledge and directly influence MCP search relevance and retrieval precision, rather than serving as promotional metrics. -#### Library synchronization (in-progress) +#### Primary collections -We'll be introducing more updates based on our library synchronization concept in upcoming releases. The base concept balances stability and currentness by integrating core guidelines and standards directly into the server while syncing from the latest available PatternFly implementation. -- **Baseline data**: Core guidelines and standards integrated directly into the server for standalone purposes, quick starts, and immediate access. -- **Dynamic content**: Content synced from the latest available PatternFly implementation while you work, ensuring the LLM always has access to the latest documentation and patterns. +- **`patternfly-docs`**: Curated Markdown documentation and guidelines catalog (`src/docs.json`) aggregating pinned upstream repositories (`patternfly-org`, `patternfly-react`, `ai-helpers`, `uxd-ai-helpers`, `patternfly-cli`, `patternfly-elements`, `patternfly-mcp`, and `pf-codemods`). +- **`patternfly-component-schemas`**: Machine-readable component JSON schemas (`@patternfly/patternfly-component-schemas`) providing runtime prop definitions and validation rules. +- **`patternfly-api`**: Live-crawled and pre-built component API specifications from PatternFly documentation endpoints. -### Configuration and Experimental Features +#### Support collections -The server utilizes a centralized **Option Registry** to handle programmatic and CLI configurations. This registry manages stability by isolating new capabilities behind `experimental` flags, allowing for rapid iteration of context management and persistence features. +- **`ai-handbook`**: Specialized Red Hat Unified Intelligence Engineering (UIE) design standards for AI experiences, fetched dynamically from [`rh-uxd/ai-handbook`](https://github.com/rh-uxd/ai-handbook). -### Data sources and integrations +#### Collection architecture -The PatternFly MCP server aggregates content from multiple official sources to provide a comprehensive development resource. +- **Multi-collection partitioning**: Records are partitioned into distinct collections within the unified library, preventing namespace collisions and enabling targeted retrieval across documentation types. +- **Worker pool and background synchronization**: Background collection processing runs on dedicated worker pools and periodic schedules, with resilience fallback ensuring server availability during transient network issues. -#### PatternFly ai-helpers -The server integrates the [patternfly/ai-helpers](https://github.com/patternfly/ai-helpers) repository to provide specialized, LLM-optimized guidance. This integration powers several key resource categories: -- **AI Guidance**: Specialized patterns for React Charts, Chatbot, and general React development. -- **Styling Standards**: CSS and styling requirements tailored for AI code generation. -- **Prompt Engineering**: Includes `ai-prompt-guidance.md` to help users write more effective prompts for PatternFly. +### Configuration and experimental features -These helpers are a core part of our [Library synchronization](#library-synchronization-in-progress), acting as the bridge between stable design patterns and dynamic implementation details. +The server utilizes a centralized **Option Registry** to handle programmatic and CLI configurations. This registry manages stability by isolating new capabilities behind `experimental` flags, allowing for rapid iteration of context management and persistence features. ### Tools, resources, and prompts as customizable plugins @@ -57,7 +52,7 @@ this actively plays a role in the library architecture because it allows us to f Key goals aided by moving towards plugins: - **Providing a tailored experience for users** - Plugins a designer uses may differ from those of a developer, researcher, or community member. -- **Evolving/future-proofing** - Plugins can evolve over time, and the MCP server can evolve to support them. (e.g., a new JS framework or design framework) +- **Evolving/future-proofing** - Plugins can evolve over time, and the MCP server can evolve to support them (e.g., a new JS framework or design framework). - **Maintainability** - MCP server core can focus on features and issues while plugins are added and maintained by the community. ## Server architecture @@ -95,20 +90,29 @@ flowchart TD Our roadmap focuses on expanding the server's reach and providing a more integrated user experience. #### In-progress -- **Experimental Context Management**: Streamlining MCP resources into two primary types: **collections** and **records**. This includes providing exact hashes for records to ensure context stability. -- **PatternFly API Integration**: Transitioning to the unified Library concept for standalone purposes and latest PatternFly version access. - - **SQLite Persistence Layer (Opt-in)**: Leveraging **Node.js 22+** to provide an optional persistence layer for up-to-date library records and resource caching. - - **Record Seed Integration**: A "fallback" set of resource records applied to every PatternFly MCP server instance that ensures users who do not opt into SQLite persistence still receive up-to-date documentation within an average MCP server use session. + +- **Dynamic MCP collection and resource invalidation and change notifications**: Emitting `sendResourceListChanged` notifications to connected MCP clients and invalidating memoized resource index caches when asynchronous or optional collections finish hydrating. +- **SQLite persistence layer (opt-in)**: Leveraging **Node.js 22+** built-in SQLite capabilities to provide an optional persistence layer for up-to-date library records and fast server startup. +- **PatternFly API crawler hardening**: Crawler throughput controls, quality filtering, and incremental crawl resume. #### In-planning and under review + +- **Resource-Tool integration**: Moving experimental MCP resources used under `experimental-context-management` into stable tooling; converting two MCP tools into a single MCP tool. +- **Collection indexing**: Indexing collections, record quality filtering, and incremental indexing. +- **Focused MCP resources**: Moving the current MCP resources to two primary resources: collections and records. - **Skills-as-Tools (On Track)**: Expand MCP functionality with agent skills using common Markdown. This provides consumers with significant customization without modifying the PatternFly MCP server core. You can start contributing to the MCP now by adding skills through our [AI Plugin Marketplace](https://github.com/rh-uxd/ai-helpers). -- **Resource-Tool Integration**: Directly integrate MCP resources into tool responses to reduce token counts and allow tools to accept URI links as inputs. -- **Environment & Analysis Tooling**: A built-in tool falling under "use PatternFly", focused on environment snapshots, code analysis, and whitelisted resource access for local project analysis. -- **Agentless MCP Client**: An MCP client for use without an LLM, allowing PatternFly tooling to integrate into CLI tools and CI/CD pipelines. -- **Resource/Helper Sharing**: Mechanisms to share resources and helper functions across external tool plugins. +- **Environment & analysis tooling**: A built-in tool falling under "use PatternFly", focused on environment snapshots, code analysis, and whitelisted resource access for local project analysis. +- **Resource and helper sharing**: Mechanisms to share resources and helper functions across external tool plugins. + +#### Future concepts + +- **Request-scoped session credentials**: Utilizing `AsyncLocalStorage` for credential passing to support private or token-gated collection sources. +- **Collection priority and dynamic overrides**: Introducing priority and grouping configurations for sorting and selective record overrides across overlapping collections. #### Deprioritized concepts and planning -- ~~**YAML Configuration**: Remote tool, resource, and prompt plugins configured via YAML.~~ Currently, superseded by Skills-as-Tools and [AI Plugin Marketplace](https://github.com/rh-uxd/ai-helpers). + +- **Agentless MCP client**: An MCP client for use without an LLM, allowing PatternFly tooling to integrate into CLI tools and CI/CD pipelines. +- ~~**YAML Configuration**: Remote tool, resource, and prompt plugins configured via YAML.~~ Currently superseded by Skills-as-Tools and [AI Plugin Marketplace](https://github.com/rh-uxd/ai-helpers). > **Contribution alignment** > diff --git a/docs/development.md b/docs/development.md index 05751c57..2917730d 100644 --- a/docs/development.md +++ b/docs/development.md @@ -155,7 +155,7 @@ const server: PfMcpInstance = await start(options); #### About pinned documentation sources -The documentation catalog `src/docs.json` pins remote resources to specific commit SHAs (or explicit refs) for stability and reproducibility. This avoids unexpected upstream changes from breaking results. The `searchPatternFlyDocs` tool handles these lookups transparently for the user. +The documentation collection `src/docs.json` pins remote resources to specific commit SHAs (or explicit refs) for stability and reproducibility. This avoids unexpected upstream changes from breaking results. The `searchPatternFlyDocs` tool handles these lookups transparently for the user. #### Programmatic runtime requirements @@ -450,18 +450,79 @@ These terms describe **how tools and their related properties are represented** For information on build maintenance, refer to [Maintenance in CONTRIBUTING.md](../CONTRIBUTING.md#nodejs-engine-bumps). -### Updating collections +### How collection and source data get updated -The server packages pre-built collections (such as `src/collection.patternFlyApi.json`) to provide quick MCP startups. +The server maintains multiple record collections that provide documentation and schemas to a consuming LLM. Each collection source follows its own maintenance lifecycle: -To refresh and validate the embedded API collection: +#### 1. Embedded documentation collection (`src/docs.json`) -```bash -npm run build:collections +> A large percentage of `docs.json` records are being moved into the PatternFly API. In the near future, `docs.json` will be renamed and incorporated into a support collection for one-off records. + +The original curated Markdown collection. It pins remote repository URLs to specific Git commit SHAs to guarantee consistent documentation and remove upstream breaking changes. + +- **Update workflow**: Follow the [add-docs-links skill](../guidelines/skills/add-docs-links/SKILL.md) to add, update, or remove entries. +- **Validation**: Ensure all raw URLs are HTTPS, match the domain whitelist in `src/options.defaults.ts`, and return 2xx responses. +- **Testing**: Run `npm test` and update `baseHashes` and the Repository Breakdown table in `src/__tests__/docs.json.test.ts` whenever refs change. + +#### 2. Pre-built API collection (`src/collection.patternFlyApi.json`) + +The server packages pre-seeded API endpoints to enable instant server startup without network overhead. + +- **Update command**: + ```bash + npm run build:collections + ``` +- **Execution**: Crawls live PatternFly API endpoints, filters quality records, updates `src/collection.patternFlyApi.json`, and executes collection-specific tests. +- **When to run**: When PatternFly publishes new releases, when updating metadata/quality filters, or as part of a general maintenance cycle. + +#### 3. Component schemas (`@patternfly/patternfly-component-schemas`) + +> A large percentage of Component schemas have been moved into the PatternFly API and are also being considered for incorporation into PatternFly React architecture and may be completely superseded in the future. + +Component JSON schemas provide runtime prop definitions, default values, and type validations. + +- **Update workflow**: Synchronized via standard npm package dependency updates (`package.json`). +- **Validation**: Verified through `src/collection.patternFlySchemas.ts` and automated unit tests. +- **When to run**: When [PatternFly Component Schemas repo](https://github.com/patternfly/patternfly-component-schemas) publishes a new release. + +#### 4. Red Hat AI Handbook (`src/collection.aiHandbook.ts`) + +> The AI Handbook is a support collection since its content goes beyond PatternFly towards AI-specific patterns at an organizational and enterprise level. + +Collection of specialized guides and AI patterns that are fetched dynamically at runtime. + +- **Update workflow**: Synchronized on runtime from the [Red Hat AI Handbook repo](https://github.com/rh-uxd/ai-handbook). +- **Validation**: Verified through automated unit tests. + +### Collection authoring and specification + +Internal and custom collections implement the `McpCollection` tuple structure: + +```typescript +type McpCollection = [ + name: string, + config: { title?: string }, + handler: (arg?: unknown) => McpCollectionResult | Promise, + _config?: { + initial?: McpCollectionResult | (() => McpCollectionResult | Promise); + runParallel?: `#${string}`; + runSchedule?: { intervalMs?: number; cancelMs?: number; delayStartMs?: number; continueOnError?: boolean; repeat?: number; }; + retainLastViable?: boolean | ((context) => boolean | Promise); + isRequired?: boolean; + } +]; ``` -- **Execution**: Crawls live PatternFly API endpoints, filters quality records, updates `src/collection.patternFlyApi.json`, and executes collection-specific Jest validation tests (`jest --selectProjects collections`). -- **When to run**: When PatternFly publishes new component API releases or when updating metadata/quality filters. +#### Tuple configuration options + +- **`name`** (`string`): Unique identifier for the collection (e.g., `'patternfly-docs'`). +- **`config`** (`object`): Plugin-visible metadata, such as `title`. +- **`handler`** (`function`): Sync, or async, function returning an `McpCollectionResult` object containing `{ records: [...] }`. +- **`_config.initial`**: Synchronous or async initial data loader executed at server startup ($t=0$) before scheduled tasks or worker threads run. +- **`_config.runParallel`**: Subpath import specifier (`#specifier`) used to execute the collection handler in a background worker pool thread (`server.workerPool`). +- **`_config.runSchedule`**: Configuration for recurring background refresh intervals via `deferTask`. +- **`_config.retainLastViable`**: Retains previously loaded valid records if an update attempt fails or returns an empty payload. +- **`_config.isRequired`**: Controls whether server startup requires this collection to be populated. ## In-progress and future work diff --git a/docs/usage.md b/docs/usage.md index d53857c0..17848ca1 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -24,8 +24,8 @@ Use this to search for PatternFly documentation URLs, `patternfly://` resource U > **Transitional URI support**: The default tools also return and accept `patternfly://` URIs for compatibility. Using and passing URIs through these tools is supported as a compatibility bridge for the intended workflow; see [experimental context management](./experimental.md#contextmanagement) for details on the transitional allowance for limited MCP clients. **Parameters:** -- `searchQuery`: `string` (required) - Case-insensitive query for full or partial keywords, resource names, versions and more (e.g., `"button"`, `"card v6"`, `"react"`, `"*"`) -- `collection`: `string` (optional) - Filter results by a primary collection of records (e.g., `"patternfly-docs"`, `"patternfly-component-schemas"`, `"patternfly-api"`). Note that additional specialized or experimental collections outside the primary core set may also be available or shift across releases. +- `searchQuery`: `string` (required) - Case-insensitive query for keywords, component names, or versions (e.g., `"button"`, `"card v6"`, `"react"`, `"*"`) +- `collection`: `string` (optional) - Filter results by a specific collection (e.g., `"patternfly-docs"`, `"patternfly-component-schemas"`, `"patternfly-api"`, `"ai-handbook"`). Note that additional specialized or experimental collections outside the primary core set may also be available or shift across releases. **Examples:** @@ -85,7 +85,7 @@ Fetch full documentation and component JSON schemas for specific PatternFly URLs The server exposes a resource-centric architecture via the `patternfly://` URI scheme. MCP clients can use these resources directly. [Review the roadmap for future resource updates](./architecture.md#roadmap). -> **Note on AI content**: Specialized AI guidance resources are sourced from the [patternfly/ai-helpers](https://github.com/patternfly/ai-helpers) integration. These are specifically optimized to help LLMs generate more accurate PatternFly code. [See Data sources and integrations in architecture](./architecture.md#data-sources-and-integrations). +> **Note on AI content**: Specialized AI guidance and engineering patterns are sourced from support collections like [rh-uxd/ai-handbook](https://github.com/rh-uxd/ai-handbook). [See Collections and data sources in architecture](./architecture.md#collections-and-data-sources). ### Discovery resources @@ -119,7 +119,7 @@ Most MCP clients use JSON configuration to specify how the server is started. Be Depending on your environment, you may have to delay updating to the minimum Node.js version required by the server. If you are unable to upgrade your Node.js version and must remain on a previous Node.js version, you can pin your MCP configuration to the last compatible version of the server. -> **Note**: Currently, pinning to an older PatternFly MCP version means you will not receive updated documentation or new features until you "update" your pinned version. In the future, pinning a version may still make an allowance for documentation updates. [See our planned architecture.](./architecture.md#library-synchronization-in-progress) +> **Note**: Currently, pinning to an older PatternFly MCP version means you will not receive updated documentation or new features until you "update" your pinned version. In the future, pinning a version may still make an allowance for documentation updates. [See our planned architecture.](./architecture.md#collections-and-data-sources) #### When to choose `@latest` or a pinned version for configuration