Skip to content

feat(mcp): surface imported team metadata, timestamps, and original_id in get-tool projections - #953

Merged
mvkonchits-db merged 1 commit into
developmentfrom
feat/mcp-ownership-visibility
Oct 12, 2026
Merged

mvkonchits-db merged 1 commit into
developmentfrom
feat/mcp-ownership-visibility

Conversation

@larsgeorge-db

Copy link
Copy Markdown
Collaborator

Summary

The Ask Ontos agent couldn't answer "who is the technical owner for data product X?" for imported ODPS products: the MCP get_data_product / get_data_contract projections returned only the native Databricks owner-team reference and dropped the imported ODPS/ODCS team block entirely — even though the UI product-detail page shows an "ODPS Team Metadata (N members)" panel reading the exact same source data. The two bulk variants from #922 (get_data_products, get_data_contracts) shared the gap through the common _serialize_* helper.

Two adjacent fields were also dropped:

  • Lifecycle timestamps — created_at / updated_at / published_at / product_created_ts. Agents frequently need "when was this registered / last updated / published?" alongside ownership.
  • Preserved YAML id — ontosOriginalId custom property, set by feat(import): adopt YAML entity id (UUID) on ODCS/ODPS import to preserve links #853 when adopt_ids=false or the YAML id was invalid/colliding. Lets an agent reconcile the Ontos UUID with an upstream reference without a second call.

Changes

  • _serialize_product (src/backend/src/tools/data_products.py): project team (name, description, members[].{username,name,role}), four lifecycle timestamps (ISO-serialized via a _iso_or_none helper), and original_id. dateIn/dateOut/replacedByUsername are intentionally omitted — not useful for ownership answers and would inflate payloads.
  • _serialize_contract (src/backend/src/tools/data_contracts.py): mirror for contracts. ODCS team is a flat List[TeamMember] (not wrapped in a Team object like products) and customProperties is a Dict[str, Any] (not a list of CustomProperty objects) — both handled explicitly. Contract timestamps are already ISO strings on DataContractRead, so no re-serialization needed.
  • Tool-description strings on GetDataProductTool / GetDataProductsBulkTool / GetDataContractTool / GetDataContractsBulkTool: tell the agent the response now includes imported team members with roles, lifecycle timestamps, and preserved original id. Agents choose tools from these descriptions, so this is important.

Scope boundary

This is Phase 1 of a two-part fix (see ~/.claude/plans/stateless-whistling-teacup.md). The complementary search coverage — indexing role and adding team-member matching for the free-text search_data_* tools, plus contract parity — is deferred to a follow-up PR stacked on #921 to keep this change independent and easy to review. Original-ID search coverage will land there too.

Tests

New: src/backend/src/tests/unit/test_mcp_tool_projections.py — 11 unit tests pinning the projection contract for products and contracts: team projection with roles, no-team case, malformed-member case (only username), timestamp (de)serialization, and ontosOriginalId lookup for both the list-of-CustomProperty shape (products) and the dict shape (contracts).

pytest backend/src/tests/unit/test_mcp_tool_projections.py \
       backend/src/tests/unit/test_mcp_bulk_get.py \
       backend/src/tests/unit/test_mcp_contracts_db_backed.py
→ 22 passed

Test plan

  • Call get_data_product on an imported ODPS fixture via the MCP client — assert team.members[*].role and the four timestamps appear.
  • Call get_data_products (bulk) with 2–3 ids — shape identical to single-get (both call _serialize_product).
  • Re-run the Ask Ontos agent reproducer ("who is the technical owner for <fixture>?") — expect the imported team members with roles to appear in the answer.
  • Same three checks for contracts: get_data_contract, get_data_contracts, agent query with ODCS fixture.
  • Confirm original_id is populated for a product imported with adopt_ids=false and None otherwise.

This pull request and its description were written by Isaac.

…in get-tool projections

The Ask Ontos agent couldn't answer "who is the technical owner for data
product X?" for imported ODPS products: the MCP `get_data_product` /
`get_data_contract` projections returned only the native Databricks owner
team reference and dropped the imported ODPS/ODCS `team` block entirely —
even though the UI product-detail page shows an "ODPS Team Metadata (N
members)" panel reading the exact same source data. The two bulk variants
(`get_data_products`, `get_data_contracts` from #922) shared the gap via
the common `_serialize_*` helper.

Also dropped: lifecycle timestamps (`created_at` / `updated_at` /
`published_at` / `product_created_ts`) and the preserved YAML id
(`ontosOriginalId` custom property, set by #853 when `adopt_ids=false` or
the YAML id was invalid/colliding). Agents frequently need both — the
first for "when was this registered / last updated?", the second to
reconcile the Ontos UUID with an upstream reference.

Changes:
- `_serialize_product` (`src/backend/src/tools/data_products.py`): project
  `team` (name, description, members[].{username,name,role}), four
  lifecycle timestamps (ISO-serialized), and `original_id`. Omit
  dateIn/dateOut/replacedByUsername — not useful for ownership answers
  and would just inflate payloads.
- `_serialize_contract` (`src/backend/src/tools/data_contracts.py`): mirror
  for contracts. ODCS `team` is a flat `List[TeamMember]` (not wrapped in
  a `Team` object like products) and `customProperties` is a dict (not a
  list of CustomProperty objects) — handled explicitly. Contract
  timestamps are already ISO strings on `DataContractRead` so no
  re-serialization needed.
- Tool-description strings on `GetDataProductTool`,
  `GetDataProductsBulkTool`, `GetDataContractTool`,
  `GetDataContractsBulkTool`: tell the agent the response includes the
  imported team metadata, lifecycle timestamps, and preserved original id.
- `test_mcp_tool_projections.py`: 11 unit tests covering the team/role
  projection, no-team case, malformed-member case, timestamp
  (de)serialization, and `ontosOriginalId` lookup for both products and
  contracts.

Session: claude -r 171078d5-61c9-42b9-aeb6-5d378c386f35

Co-authored-by: Isaac <no-reply@databricks.com>
@larsgeorge-db
larsgeorge-db requested a review from a team October 11, 2026 10:22
@larsgeorge-db
larsgeorge-db requested a review from a team as a code owner October 11, 2026 10:22
larsgeorge-db added a commit that referenced this pull request Oct 11, 2026
@larsgeorge-db larsgeorge-db added the stakeholder/fpiz Reported by external stakeholder (codename: fpiz) label Oct 11, 2026
larsgeorge-db added a commit that referenced this pull request Oct 11, 2026

@mvkonchits-db mvkonchits-db left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. I checked the new fields against the models and builders, not just the diff.

  • Every field read here exists with the type the code expects: DataProduct.team (a Team with members), the four datetime fields, DataContractRead.team as a flat list, customProperties as a dict for contracts and a list for products, and the string timestamps on contracts.
  • _build_contract_api_model really populates team, customProperties, created, updated and published_at, so the new fields are not silently empty.
  • Custom property string values are stored verbatim on both sides, so original_id comes back without extra quotes.
  • CI is green and the PR merges cleanly with #954 and #921.

Please look at these two before or soon after merge:

  1. Former members will look like current owners. dateOut and replacedByUsername are left out on purpose, but the tool exists to answer "who is the technical owner?". A member with a past dateOut, or one who was replaced, is returned as if still active. Adding date_out and replaced_by_username to each member is cheap and lets the agent answer correctly. (The same gap exists in the search index in #921.)
  2. The original_id description is wrong. The PR text and the tool descriptions say it is set "when adoption was off". In the code ontosOriginalId is written for every import whose file had an id, including when the id was adopted (then it is just the id, as written in the file). So the test plan line "None otherwise" will not hold, and the agent is told something untrue. Older imports only have the legacy sourceId, so they return null. Suggest wording like "the id as written in the source file, when it had one".

Smaller notes, none blocking:

  • The 11 unit tests use SimpleNamespace stand-ins, so a field rename on the real models would not break them. One test built from a real DataProduct and DataContractRead would pin the contract.
  • team is None for products and [] for contracts. Fine if intended, but worth one line in the descriptions.
  • Member usernames (often emails) now go to any token with data-products:read or contracts:read. The UI already shows the same data, but the MCP get tools do not apply per-user visibility (true since #939), so a draft that only its owner can see is returned to any token holder. Worth stating in the design notes.
  • One bad timestamp value makes the whole bulk call fail, not just that item.

@mvkonchits-db
mvkonchits-db added this pull request to the merge queue Oct 12, 2026
Merged via the queue into development with commit 05d4952 Oct 12, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stakeholder/fpiz Reported by external stakeholder (codename: fpiz)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants