Skip to content

chat: use structured outputs for SearchBookContent MCP tool#1079

Merged
BenjaminMichaelis merged 1 commit into
mainfrom
agents/ai-chat-structured-output-review
May 16, 2026
Merged

chat: use structured outputs for SearchBookContent MCP tool#1079
BenjaminMichaelis merged 1 commit into
mainfrom
agents/ai-chat-structured-output-review

Conversation

@BenjaminMichaelis
Copy link
Copy Markdown
Member

@BenjaminMichaelis BenjaminMichaelis commented May 9, 2026

Summary

Improves structured output consistency across the AI chat RAG pipeline by converting SearchBookContent — the primary semantic vector search tool — to use the same hybrid structured content pattern already established by FindBookHelpForDiagnostic and SearchListingsByCode.

Changes

SearchBookContent converted to structured output

SearchBookContent was the only major RAG tool still returning plain text. It now returns a CallToolResult with:

  • UseStructuredContent = true and OutputSchemaType = typeof(SearchBookContentResult)
  • McpToolResultFormatter.GetModelInput() will now return the JSON StructuredContent to the model instead of the formatted text, consistent with all other structured tools
  • The human-readable text is preserved as the first Content block for MCP clients that display it

"No results" semantics fixed

The zero-results path previously returned McpToolResultFactory.CreateError() (IsError = true). Zero results from a search query is a valid outcome, not a tool failure. It now returns CreateHybridResult with an empty Matches: [] list, so the model always receives well-typed JSON conforming to the schema.

Validation failures (empty query, query too long, AI service not configured) remain correctly modeled as errors.

New structured result types

Added to McpToolResults.cs:

  • SearchBookContentMatchResult — one vector search hit (score, chapter number, heading, text chunk)
  • SearchBookContentResult — wrapper with Matches list; this is the advertised output schema

OutputSchemaType made explicit on all UseStructuredContent tools

Five tools that had UseStructuredContent = true but no OutputSchemaType were updated:
GetChapterList, GetChapterSections, GetDirectContentUrl, GetNavigationContext, GetChapterSummary.

The MCP SDK already auto-infers the output schema from the concrete return type (confirmed by pre-existing integration tests), so this is redundant but makes the intended schema contract explicit at the declaration site — consistent with how FindBookHelpForDiagnostic and SearchListingsByCode are declared.

Contract test updated

McpToolContractTests.McpToolsList_StructuredAndHybridTools_AdvertiseOutputSchema now asserts that search_book_content advertises an outputSchema, alongside the other structured tools.

What was NOT changed (and why)

  • Main chat response formatResponseTextFormat.CreateJsonSchemaFormat() does not exist in OpenAI SDK 2.7.0 for the Responses API. Schema-enforced output on the streaming text response isn't possible without switching to the Chat Completions API, and would break markdown rendering anyway. Both subagents agreed this is correct.
  • strictModeEnabled: true on tool input registration — Already correct in AIChatService.cs, untouched.
  • Plain-text tools (LookupConcept, CheckTopicCoverage, FindRelatedSections, etc.) — These return prose-heavy navigation summaries where structured content wouldn't add value for the model.

Testing

  • ✅ Solution builds: 0 errors, 0 warnings
  • ✅ All 11 chat unit tests pass
  • ✅ Contract test updated to cover search_book_content schema advertisement

- Convert SearchBookContent from Task<string> to Task<CallToolResult> with
  UseStructuredContent = true and OutputSchemaType = typeof(SearchBookContentResult),
  matching the pattern established by FindBookHelpForDiagnostic and SearchListingsByCode
- Add SearchBookContentMatchResult and SearchBookContentResult records to McpToolResults.cs
- Fix no-results path to return CreateHybridResult with empty Matches list instead of
  CreateError — zero search results is a valid outcome, not a tool failure
- Add explicit OutputSchemaType to GetChapterList, GetChapterSections, GetDirectContentUrl,
  GetNavigationContext, and GetChapterSummary for documentation clarity (MCP SDK already
  infers the output schema from the concrete return type, but explicit annotation makes
  the intended schema contract visible at the declaration site)
- Add search_book_content to McpToolContractTests outputSchema presence assertions
@BenjaminMichaelis BenjaminMichaelis marked this pull request as ready for review May 16, 2026 02:22
Copilot AI review requested due to automatic review settings May 16, 2026 02:22
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Converts the SearchBookContent MCP tool to return structured (hybrid) output, aligning it with the other RAG tools (FindBookHelpForDiagnostic, SearchListingsByCode). Also fixes "no results" semantics so a zero-match search is no longer modeled as a tool error, and makes OutputSchemaType explicit on all UseStructuredContent tools for declarative clarity.

Changes:

  • SearchBookContent now returns a CallToolResult via CreateHybridResult with a SearchBookContentResult structured payload; zero-results returns an empty Matches list instead of IsError = true.
  • Added new result records SearchBookContentMatchResult / SearchBookContentResult to McpToolResults.cs.
  • Added explicit OutputSchemaType on five existing structured tools (GetChapterList, GetChapterSections, GetDirectContentUrl, GetNavigationContext, GetChapterSummary) and extended the contract test to assert search_book_content advertises an output schema.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File Description
EssentialCSharp.Web/Tools/BookSearchTool.cs Converts SearchBookContent to hybrid structured result; adds explicit OutputSchemaType to several tool attributes.
EssentialCSharp.Web/Tools/BookContentTool.cs Adds explicit OutputSchemaType to GetNavigationContext and GetChapterSummary.
EssentialCSharp.Web/Models/McpToolResults.cs Introduces SearchBookContentMatchResult and SearchBookContentResult records for the new schema.
EssentialCSharp.Web.Tests/McpToolContractTests.cs Asserts that search_book_content advertises an outputSchema.

@BenjaminMichaelis BenjaminMichaelis merged commit 73868a6 into main May 16, 2026
11 checks passed
@BenjaminMichaelis BenjaminMichaelis deleted the agents/ai-chat-structured-output-review branch May 16, 2026 02:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants