-
-
Notifications
You must be signed in to change notification settings - Fork 1.1k
feat(examples): add OpenUI PDF RAG integration #1401
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
d3b89d8
7ed0f55
4d678fe
711f8e5
0b6eba2
cf440c3
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| OPENAI_API_KEY=your-openai-api-key | ||
| VOLTAGENT_MODEL=openai/gpt-4o-mini | ||
| VOLTAGENT_EMBEDDING_MODEL=text-embedding-3-small |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| /node_modules | ||
| /.next | ||
| /out | ||
| /coverage | ||
| /openui/generated | ||
| /.env* | ||
| !/.env.example | ||
| *.tsbuildinfo | ||
| next-env.d.ts |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,144 @@ | ||
| # VoltAgent + OpenUI PDF RAG | ||
|
|
||
| This example connects [VoltAgent](https://voltagent.dev/) to | ||
| [OpenUI](https://openui.com/) in a grounded, streaming RAG chat. VoltAgent | ||
| extracts and chunks one bundled PDF, embeds the chunks in memory, retrieves the | ||
| most relevant pages for every turn, and supplies that context to the model. | ||
| OpenUI turns the grounded answer into interactive charts, forms, and follow-up | ||
| actions. | ||
|
|
||
| The bundled [2025 Housing Supply Report](./data/nyc-2025-housing-supply-report.pdf) | ||
| from the [New York City Rent Guidelines Board](https://rentguidelinesboard.cityofnewyork.us/research/) | ||
| is the only runtime source of housing facts. It is an official annual government | ||
| report covering calendar-year 2024. The starter prompts contain no hidden chart | ||
| values, and there is no fallback mock dataset. The bundled file was downloaded | ||
| from the Board's [official PDF](https://rentguidelinesboard.cityofnewyork.us/wp-content/uploads/2025/05/2025-HSR.pdf). | ||
|
|
||
| ## Supported versions | ||
|
|
||
| The example is tested from this repository lockfile with VoltAgent | ||
| `@voltagent/core` 2.9.x, `@voltagent/rag` 1.0.x, `pdf-parse` 2.4.x, and OpenUI | ||
| `@openuidev/react-ui` 0.13.x, `@openuidev/react-lang` 0.2.x, and | ||
| `@openuidev/react-headless` 0.9.x. | ||
|
|
||
| ## Run it | ||
|
|
||
| From the repository root: | ||
|
|
||
| ```bash | ||
| pnpm install | ||
| cp examples/with-openui/.env.example examples/with-openui/.env | ||
| # Add your OPENAI_API_KEY to examples/with-openui/.env | ||
| pnpm --filter voltagent-example-with-openui dev | ||
| ``` | ||
|
|
||
| Open [http://localhost:3000](http://localhost:3000). `VOLTAGENT_MODEL` defaults | ||
| to `openai/gpt-4o-mini`, and `VOLTAGENT_EMBEDDING_MODEL` defaults to | ||
| `text-embedding-3-small`. Both accept bare OpenAI model IDs; the chat setting | ||
| also accepts an `openai/` prefix. | ||
|
|
||
| The API key, PDF parsing, embeddings, and retrieval all stay server-side. Do | ||
| not expose the key with a `NEXT_PUBLIC_` prefix. | ||
|
|
||
| ## Architecture | ||
|
|
||
| ```text | ||
| data/nyc-2025-housing-supply-report.pdf | ||
| -> pdf-parse page extraction (once, lazily) | ||
| -> @voltagent/rag RecursiveChunker | ||
| -> OpenAI embeddings cached in memory | ||
|
|
||
| AgentInterface user/action | ||
| -> POST /api/chat with the full thread and threadId | ||
| -> HttpOnly anonymous visitor cookie + server-derived conversation ID | ||
| -> VoltAgent Agent.streamText() | ||
| -> NycHousingPdfRetriever vector search | ||
| -> retrieved page context injected by VoltAgent | ||
| -> model emits source-grounded OpenUI Lang | ||
| -> OpenAI-compatible SSE | ||
| -> OpenUI adapter -> parser -> AgentInterface renderer | ||
| ``` | ||
|
|
||
| `voltagent/pdf-retriever.ts` extends VoltAgent's `BaseRetriever`. Its index is | ||
| created on the first request and reused for the life of the server process. It | ||
| ranks chunks semantically, then expands the best matches back to their complete | ||
| PDF pages so multi-column tables and map labels remain intact. VoltAgent injects | ||
| those pages, the report title, and exact page numbers into the agent's prompt as | ||
| relevant context. No vector database is required for this small, | ||
| single-document demo. Matches below the `0.30` cosine-similarity cutoff are | ||
| discarded; retrieval failures produce an explicit source-unavailable context so | ||
| the agent refuses to answer from memory. | ||
|
|
||
| `openui/library.ts` exports the same `openuiChatLibrary` used by the renderer. | ||
| The OpenUI CLI generates `openui/generated/system-prompt.txt` from that file, | ||
| and the VoltAgent agent reads the generated prompt on the server. Regenerate it | ||
| after changing the library: | ||
|
|
||
| ```bash | ||
| pnpm --filter voltagent-example-with-openui generate:prompt | ||
| ``` | ||
|
|
||
| The `AgentInterface` keeps its built-in `ThemeProvider` enabled and applies a | ||
| visible VoltAgent-green dark theme. Its built-in `ContinueConversation` action | ||
| adapter sends a `FollowUpItem` label or `@ToAssistant` form action back through | ||
| the same `/api/chat` route. Form state is serialized into the user message | ||
| context, so VoltAgent receives the values the user actually edited. | ||
|
|
||
| ## Replace the datasource | ||
|
|
||
| To use a different source of truth: | ||
|
|
||
| 1. Replace `data/nyc-2025-housing-supply-report.pdf` with your PDF. | ||
| 2. Update the report filename, title, publisher, source URL, and domain-specific agent rules. | ||
| 3. Restart the server so the lazy in-memory index is rebuilt. | ||
|
|
||
| For a larger or frequently changing document collection, move chunk embeddings | ||
| to a persistent vector store and invalidate the index when documents change. | ||
| The demo isolates in-memory conversations with a server-generated anonymous ID | ||
| stored in an HttpOnly cookie and hashes that identity with the client thread ID | ||
| before calling VoltAgent. In production, replace the anonymous identity with an | ||
| authenticated server identity, validate conversation ownership, add rate | ||
| limiting to `/api/chat`, configure durable conversation memory, and treat | ||
| uploaded PDF text as untrusted input. | ||
|
|
||
| ## Acceptance prompts | ||
|
|
||
| Chart: | ||
|
|
||
| > Using only the official 2025 Housing Supply Report PDF, show 2024 residential | ||
| > building permits for all five New York City boroughs as a labeled bar chart. | ||
| > Cite the source page and end with two relevant follow-up suggestions. | ||
|
|
||
| The chart should show Bronx `3,125`, Brooklyn `6,588`, Manhattan `2,347`, Queens | ||
| `3,240`, and Staten Island `326` dwelling units, citing PDF page 6. Those values | ||
| must come from retrieval, not the prompt. | ||
|
|
||
| Click either rendered follow-up. Its exact label should appear as one new user | ||
| turn and produce one new VoltAgent request in the same conversation. | ||
|
|
||
| Analysis: | ||
|
|
||
| > Using only the official source report, explain the most important 2024 housing | ||
| > supply and vacancy signals, including where borough trends diverged. Cite | ||
| > source pages and end with two next questions. | ||
|
|
||
| Form: | ||
|
|
||
| > Create a validated housing analysis form with required focus area and audience | ||
| > fields plus notes. Add a primary Analyze button that sends the completed | ||
| > values to you. | ||
|
|
||
| Leave required fields empty to see validation. Then submit focus area | ||
| `Vacancy rates`, audience `City planners`, and notes | ||
| `Compare borough differences`. The next rendered answer should acknowledge the | ||
| focus area and audience, then use only facts retrieved from the PDF. | ||
|
|
||
| ## Checks | ||
|
|
||
| ```bash | ||
| pnpm --filter voltagent-example-with-openui generate:prompt | ||
| pnpm --filter voltagent-example-with-openui typecheck | ||
| pnpm --filter voltagent-example-with-openui lint | ||
| pnpm --filter voltagent-example-with-openui test | ||
| pnpm --filter voltagent-example-with-openui build | ||
| ``` |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,84 @@ | ||
| import { readFileSync } from "node:fs"; | ||
| import { join } from "node:path"; | ||
| import { InvalidChatRequestError, parseChatRequestBody } from "@/lib/chat-request"; | ||
| import { deriveConversationId, resolveAnonymousSession } from "@/lib/chat-session"; | ||
| import { createOpenUIAgent } from "@/voltagent/agent"; | ||
| import { safeStringify } from "@voltagent/internal/utils"; | ||
|
|
||
| export const runtime = "nodejs"; | ||
|
|
||
| const openUISystemPrompt = readFileSync( | ||
| join(process.cwd(), "openui/generated/system-prompt.txt"), | ||
| "utf8", | ||
| ); | ||
| const agent = createOpenUIAgent(openUISystemPrompt); | ||
|
|
||
| function completionChunk(id: string, content: string) { | ||
| return `data: ${safeStringify({ | ||
| id, | ||
| object: "chat.completion.chunk", | ||
| choices: [{ index: 0, delta: { content }, finish_reason: null }], | ||
| })}\n\n`; | ||
| } | ||
|
|
||
| function stopChunk(id: string) { | ||
| return `data: ${safeStringify({ | ||
| id, | ||
| object: "chat.completion.chunk", | ||
| choices: [{ index: 0, delta: {}, finish_reason: "stop" }], | ||
| })}\n\n`; | ||
| } | ||
|
|
||
| export async function POST(request: Request) { | ||
| try { | ||
| const { messages, threadId = "openui-demo" } = await parseChatRequestBody(request); | ||
| const anonymousSession = resolveAnonymousSession(request.headers.get("cookie")); | ||
| const result = await agent.streamText(messages, { | ||
| userId: anonymousSession.userId, | ||
| conversationId: deriveConversationId(anonymousSession.userId, threadId), | ||
| abortSignal: request.signal, | ||
| }); | ||
|
|
||
| const encoder = new TextEncoder(); | ||
| const responseId = `voltagent-${crypto.randomUUID()}`; | ||
|
|
||
| const stream = new ReadableStream({ | ||
| async start(controller) { | ||
| try { | ||
| for await (const text of result.textStream) { | ||
| controller.enqueue(encoder.encode(completionChunk(responseId, text))); | ||
| } | ||
|
|
||
| controller.enqueue(encoder.encode(stopChunk(responseId))); | ||
| controller.enqueue(encoder.encode("data: [DONE]\n\n")); | ||
| controller.close(); | ||
| } catch (error) { | ||
| if (request.signal.aborted) { | ||
| return; | ||
| } | ||
|
|
||
| console.error("[voltagent-openui] stream failed", error); | ||
| controller.error(error); | ||
| } | ||
| }, | ||
| }); | ||
|
|
||
| const headers = new Headers({ | ||
| "Cache-Control": "no-cache, no-transform", | ||
| Connection: "keep-alive", | ||
| "Content-Type": "text/event-stream", | ||
| }); | ||
| if (anonymousSession.setCookie) headers.append("Set-Cookie", anonymousSession.setCookie); | ||
|
|
||
| return new Response(stream, { | ||
| headers, | ||
| }); | ||
| } catch (error) { | ||
| if (error instanceof InvalidChatRequestError) { | ||
| return Response.json({ error: "Invalid chat request" }, { status: 400 }); | ||
| } | ||
|
|
||
| console.error("[voltagent-openui] route failed", error); | ||
| return Response.json({ error: "Unable to start the VoltAgent stream" }, { status: 500 }); | ||
| } | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| @import "tailwindcss"; | ||
| @import "@openuidev/react-ui/styles/index.css"; | ||
|
|
||
| :root { | ||
| color-scheme: dark; | ||
| } | ||
|
|
||
| html, | ||
| body { | ||
| height: 100%; | ||
| margin: 0; | ||
| } | ||
|
|
||
| body { | ||
| background: oklch(0.15 0.015 162); | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| import type { Metadata } from "next"; | ||
| import "./globals.css"; | ||
|
|
||
| export const metadata: Metadata = { | ||
| title: "VoltAgent + OpenUI", | ||
| description: "Interactive generative UI streamed by VoltAgent and rendered with OpenUI", | ||
| }; | ||
|
|
||
| export default function RootLayout({ children }: Readonly<{ children: React.ReactNode }>) { | ||
| return ( | ||
| <html lang="en"> | ||
| <body>{children}</body> | ||
| </html> | ||
| ); | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,60 @@ | ||
| "use client"; | ||
|
|
||
| import { | ||
| AgentInterface, | ||
| createTheme, | ||
| fetchLLM, | ||
| openAIAdapter, | ||
| openAIMessageFormat, | ||
| } from "@openuidev/react-ui"; | ||
| import { openuiChatLibrary } from "@openuidev/react-ui/genui-lib"; | ||
| import { useMemo } from "react"; | ||
|
|
||
| const voltAgentTheme = createTheme({ | ||
| background: "oklch(0.15 0.015 162 / 1)", | ||
| foreground: "oklch(0.2 0.018 162 / 1)", | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. P2: In the dark theme, Prompt for AI agents |
||
| interactiveAccentDefault: "oklch(0.76 0.18 157 / 1)", | ||
| interactiveAccentHover: "oklch(0.82 0.17 157 / 1)", | ||
| textBrand: "oklch(0.82 0.17 157 / 1)", | ||
| }); | ||
|
|
||
| export default function Home() { | ||
| const llm = useMemo( | ||
| () => | ||
| fetchLLM({ | ||
| url: "/api/chat", | ||
| streamAdapter: openAIAdapter(), | ||
| messageFormat: openAIMessageFormat, | ||
| }), | ||
| [], | ||
| ); | ||
|
|
||
| return ( | ||
| <main className="h-screen w-screen overflow-hidden"> | ||
| <AgentInterface | ||
| llm={llm} | ||
| componentLibrary={openuiChatLibrary} | ||
| agentName="VoltAgent + OpenUI" | ||
| theme={{ mode: "dark", darkTheme: voltAgentTheme }} | ||
| starterVariant="short" | ||
| starters={[ | ||
| { | ||
| displayText: "2024 permits by borough", | ||
| prompt: | ||
| "Using only the official 2025 Housing Supply Report PDF, show 2024 residential building permits for all five New York City boroughs as a labeled bar chart. Cite the source page and end with two relevant follow-up suggestions.", | ||
| }, | ||
| { | ||
| displayText: "Explain NYC housing supply", | ||
| prompt: | ||
| "Using only the official source report, present the most important 2024 housing supply and vacancy signals as a compact dashboard with a headline takeaway, key-metrics table, and borough comparison visual. Keep prose brief, cite source pages, and end with two next questions.", | ||
| }, | ||
| { | ||
| displayText: "Housing analysis form", | ||
| prompt: | ||
| "Create a validated housing analysis form with required focus area and audience fields plus notes. Add a primary Analyze button that sends the completed values to you.", | ||
| }, | ||
| ]} | ||
| /> | ||
| </main> | ||
| ); | ||
| } | ||
Uh oh!
There was an error while loading. Please reload this page.