feat(mcp): collapse list_* into search wrappers + dedupe tags + add include_facets / fields

[larsgeorge-db]

Summary

An agent-driven test of the live MCP tool set surfaced inefficiencies — one real bug and shape issues that make every call heavier than it needs to be. This PR addresses them in one coordinated change.

1. list_data_products was broken. It called data_products_manager.list_products(skip, limit, domain=..., status=...) but the manager method doesn't accept domain or status kwargs — every call raised TypeError and returned a generic error. Even if the signatures matched, the method defaults to is_admin=False with no caller scope → fail-closed empty list. list_data_contracts didn't have the kwarg bug but hardcoded is_admin=True and filtered domain post-fetch inefficiently, and returned a different shape than its search_* sibling.

Fix: both ListData*Tool.execute() now delegate to the corresponding SearchData*Tool with query='*'. One implementation, one response shape, no bug surface. Tool descriptions tell the agent to prefer search_* for anything non-trivial (same pattern already used on get_data_products).

2. Duplicate tags in indexed items. Both managers' _build_search_index_item accumulate tags from multiple sources (unified tags, assigned domains, ontosOriginalDomain, concept labels, team members) without dedupe. The result's tags[:5] slice then wasted slots on repeats — the agent observed e.g. ["Supply Chain", "Inventory", "Inventory"].

Fix: order-preserving dedupe at the end of each builder.

3. feature_id leaked into every search result. _to_result and the Postgres _row_to_result emitted feature_id on every row. type already differentiates data-product from data-contract for MCP consumers, so this was near-redundant.

Fix: drop feature_id from both result builders.

4. Facets always computed. search_scoring.search_index(include_facets=True) already supports opting out, but the MCP tool signatures didn't expose it.

Fix: search_data_products and search_data_contracts now accept include_facets: bool = True and thread it through query_index → backend search → search_scoring / search_repository.

5. No fields / projection parameter. Every call paid for description + tags + owner + version even when the agent only needed id + name.

Fix: add fields: Optional[List[str]] to the search_* signatures, applied per-row (id always retained). Threaded end-to-end through both the in-memory and Postgres backends.

6. type and link leaked into every per-type MCP record. The shared scorer (_to_result + Postgres _row_to_result) emits type and link so the UI global search — which mixes entity types — can route and distinguish them. For an MCP agent calling a per-type tool (search_data_products / list_data_products and the contracts twins), the tool name already carries the type, and link is a UI route path with no agent-side use.

Fix: strip type and link from every record in the four per-type MCP tools, after query_index returns. Unconditional — an explicit fields=["type","link"] won't resurrect them. The shared scorer and the UI global-search path are untouched, so mixed-type results keep both fields.

7. Contract search description was a composite UI string. The contract index builder composed description as "{version} • {status} • {description_usage}". Both version and status were already stored as structured extra_data fields and surface as separate top-level keys on each search result row via _RESULT_FIELDS — the composite was pure duplication that cost tokens on every list/search call and made the description harder for MCP agents to parse. Products never did this.

Fix: store just the raw description_usage in the index. UI result cards get a cleaner description line; MCP agents get a parseable string plus the structured version/status they already had.

Tests

src/backend/src/tests/unit/test_mcp_tool_efficiency.py — 16 unit tests.

• TestListTools — list_data_products / list_data_contracts delegate to the search tool with query='*', correct type_filter, correct filter kwargs.
• TestToResult — _to_result omits feature_id; fields=[...] narrows rows (always keeps id); include_facets=False returns empty facets with records unchanged. Also pins that type + link stay at the shared scorer layer (needed by UI global search).
• TestTagDedupe — product and contract index builders dedupe tags preserving insertion order.
• TestSearchParamPassthrough — search_* passes include_facets and fields through to query_index; defaults are back-compat.
• TestPerRecordTypeAndLinkStrip — search_data_products, search_data_contracts, and their list_* thin wrappers all drop type and link from every result record.
• TestContractDescriptionIsRaw — contract index stores raw description_usage with no version • status • prefix; version/status remain available via extra_data.

Also updated: test_mcp_contracts_db_backed.py::test_list_includes_db_written_contract now builds a SearchManager over the test session so the "DB-backed list" invariant (#918) still holds through the search path.

pytest backend/src/tests/unit/test_mcp_tool_efficiency.py
       backend/src/tests/unit/test_mcp_contracts_db_backed.py
       backend/src/tests/unit/test_mcp_bulk_get.py
       backend/src/tests/unit/test_search_scoring.py
       backend/src/tests/unit/test_search_backend.py
→ green

Test plan

• Call list_data_products via the MCP client — expect no 500, response shape matches search_data_products (products / total_found / has_more / offset / limit / facets).
• Call list_data_contracts — same shape as search_data_contracts.
• Call search_data_products with include_facets=false — facets block is empty, records present.
• Call search_data_products with fields=['title','status'] — rows contain only id, title, status.
• Confirm result rows no longer contain feature_id, type, or link.
• Confirm contract search description no longer carries the version • status • prefix; version/status still surface as separate structured keys on each row.
• Confirm no duplicate strings inside a result's tags list on a product that previously duplicated.
• UI global search (not MCP) still returns type and link on each record (unchanged behaviour at the shared scorer layer).

This pull request and its description were written by Isaac.

[mvkonchits-db]

Requesting changes on one point (1). The rest are notes. The direction is good: list_data_products really was broken on development (it passes domain and status to list_products, which does not accept them, so every call raised a TypeError), and collapsing the list tools into the search tools removes a whole class of drift. I checked the points below in the code. I did not run the tests.

1. list_data_contracts can now show stale data. #923 made this tool read from the database on purpose, because the in-memory path was empty. With this PR it reads the search index instead, and the index is not updated on every status change:

• DataContractsManager.transition_status sets the status and commits, with no _update_search_index. The three REST endpoints that call it (data_contracts_routes.py around lines 225, 276 and 340) do not reindex either.
• workflow_executor sets active and deployed directly on the row, also without reindexing.

So a contract that has just become active can show up as draft in list_data_contracts, and status=active misses it until the next restart or index rebuild. search_data_contracts already has this gap today, but this PR extends it to the tool that #923 fixed. The updated test test_list_includes_db_written_contract builds the index after the write, and its docstring says so, so it cannot catch this. Please either call _update_search_index after status transitions (and in the workflow path), or keep the list tool on the database. A test that creates a contract, builds the index, then changes the status and calls the list tool would cover it.

2. The list_data_contracts response keys change. name becomes title and owner_team_name becomes owner, because it now returns search rows. The description mentions the new shape but not the renames. Nothing breaks for products (that tool never worked), but contract consumers should be told.

3. fields fails silently on a wrong name. The projection keeps only keys that exist in the row, and the row key is title, not name. fields=['name', 'status'] returns id and status only, with no error. Either validate the names and return a clear message, or accept name as an alias, since the old list tools used it.

Checked and fine:

• Dropping feature_id only affects the MCP result dicts. The frontend search uses /api/search, which returns SearchIndexItem objects and is unchanged.
• Domain and status filters stay exact and case insensitive, and the limit of 100 is enforced in both backends.
• The tag de-duplication keeps order and also reduces the duplicate tags I flagged on #921. It only dedupes exact strings, so Finance and finance stay separate.
• fields is a membership check on a fixed set of keys, so there is no injection risk.
• The PR merges cleanly with #953 and #921.

I will re-review as soon as point 1 is addressed.