Repository navigation
feat(mcp): surface imported team metadata, timestamps, and original_id in get-tool projections - #953
Merged
Conversation
…in get-tool projections The Ask Ontos agent couldn't answer "who is the technical owner for data product X?" for imported ODPS products: the MCP `get_data_product` / `get_data_contract` projections returned only the native Databricks owner team reference and dropped the imported ODPS/ODCS `team` block entirely — even though the UI product-detail page shows an "ODPS Team Metadata (N members)" panel reading the exact same source data. The two bulk variants (`get_data_products`, `get_data_contracts` from #922) shared the gap via the common `_serialize_*` helper. Also dropped: lifecycle timestamps (`created_at` / `updated_at` / `published_at` / `product_created_ts`) and the preserved YAML id (`ontosOriginalId` custom property, set by #853 when `adopt_ids=false` or the YAML id was invalid/colliding). Agents frequently need both — the first for "when was this registered / last updated?", the second to reconcile the Ontos UUID with an upstream reference. Changes: - `_serialize_product` (`src/backend/src/tools/data_products.py`): project `team` (name, description, members[].{username,name,role}), four lifecycle timestamps (ISO-serialized), and `original_id`. Omit dateIn/dateOut/replacedByUsername — not useful for ownership answers and would just inflate payloads. - `_serialize_contract` (`src/backend/src/tools/data_contracts.py`): mirror for contracts. ODCS `team` is a flat `List[TeamMember]` (not wrapped in a `Team` object like products) and `customProperties` is a dict (not a list of CustomProperty objects) — handled explicitly. Contract timestamps are already ISO strings on `DataContractRead` so no re-serialization needed. - Tool-description strings on `GetDataProductTool`, `GetDataProductsBulkTool`, `GetDataContractTool`, `GetDataContractsBulkTool`: tell the agent the response includes the imported team metadata, lifecycle timestamps, and preserved original id. - `test_mcp_tool_projections.py`: 11 unit tests covering the team/role projection, no-team case, malformed-member case, timestamp (de)serialization, and `ontosOriginalId` lookup for both products and contracts. Session: claude -r 171078d5-61c9-42b9-aeb6-5d378c386f35 Co-authored-by: Isaac <no-reply@databricks.com>
larsgeorge-db
added a commit
that referenced
this pull request
Oct 11, 2026
…ginal_id) into combined test branch
larsgeorge-db
added a commit
that referenced
this pull request
Oct 11, 2026
…ginal_id) into combined test branch
mvkonchits-db
approved these changes
Oct 12, 2026
mvkonchits-db
left a comment
Contributor
There was a problem hiding this comment.
Approving. I checked the new fields against the models and builders, not just the diff.
- Every field read here exists with the type the code expects:
DataProduct.team(aTeamwithmembers), the four datetime fields,DataContractRead.teamas a flat list,customPropertiesas a dict for contracts and a list for products, and the string timestamps on contracts. _build_contract_api_modelreally populatesteam,customProperties,created,updatedandpublished_at, so the new fields are not silently empty.- Custom property string values are stored verbatim on both sides, so
original_idcomes back without extra quotes. - CI is green and the PR merges cleanly with #954 and #921.
Please look at these two before or soon after merge:
- Former members will look like current owners.
dateOutandreplacedByUsernameare left out on purpose, but the tool exists to answer "who is the technical owner?". A member with a pastdateOut, or one who was replaced, is returned as if still active. Addingdate_outandreplaced_by_usernameto each member is cheap and lets the agent answer correctly. (The same gap exists in the search index in #921.) - The
original_iddescription is wrong. The PR text and the tool descriptions say it is set "when adoption was off". In the codeontosOriginalIdis written for every import whose file had an id, including when the id was adopted (then it is just the id, as written in the file). So the test plan line "Noneotherwise" will not hold, and the agent is told something untrue. Older imports only have the legacysourceId, so they returnnull. Suggest wording like "the id as written in the source file, when it had one".
Smaller notes, none blocking:
- The 11 unit tests use
SimpleNamespacestand-ins, so a field rename on the real models would not break them. One test built from a realDataProductandDataContractReadwould pin the contract. teamisNonefor products and[]for contracts. Fine if intended, but worth one line in the descriptions.- Member usernames (often emails) now go to any token with
data-products:readorcontracts:read. The UI already shows the same data, but the MCP get tools do not apply per-user visibility (true since #939), so a draft that only its owner can see is returned to any token holder. Worth stating in the design notes. - One bad timestamp value makes the whole bulk call fail, not just that item.
Open
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The Ask Ontos agent couldn't answer "who is the technical owner for data product X?" for imported ODPS products: the MCP
get_data_product/get_data_contractprojections returned only the native Databricks owner-team reference and dropped the imported ODPS/ODCSteamblock entirely — even though the UI product-detail page shows an "ODPS Team Metadata (N members)" panel reading the exact same source data. The two bulk variants from #922 (get_data_products,get_data_contracts) shared the gap through the common_serialize_*helper.Two adjacent fields were also dropped:
created_at/updated_at/published_at/product_created_ts. Agents frequently need "when was this registered / last updated / published?" alongside ownership.ontosOriginalIdcustom property, set by feat(import): adopt YAML entity id (UUID) on ODCS/ODPS import to preserve links #853 whenadopt_ids=falseor the YAML id was invalid/colliding. Lets an agent reconcile the Ontos UUID with an upstream reference without a second call.Changes
_serialize_product(src/backend/src/tools/data_products.py): projectteam(name, description,members[].{username,name,role}), four lifecycle timestamps (ISO-serialized via a_iso_or_nonehelper), andoriginal_id.dateIn/dateOut/replacedByUsernameare intentionally omitted — not useful for ownership answers and would inflate payloads._serialize_contract(src/backend/src/tools/data_contracts.py): mirror for contracts. ODCSteamis a flatList[TeamMember](not wrapped in aTeamobject like products) andcustomPropertiesis aDict[str, Any](not a list ofCustomPropertyobjects) — both handled explicitly. Contract timestamps are already ISO strings onDataContractRead, so no re-serialization needed.GetDataProductTool/GetDataProductsBulkTool/GetDataContractTool/GetDataContractsBulkTool: tell the agent the response now includes imported team members with roles, lifecycle timestamps, and preserved original id. Agents choose tools from these descriptions, so this is important.Scope boundary
This is Phase 1 of a two-part fix (see
~/.claude/plans/stateless-whistling-teacup.md). The complementary search coverage — indexingroleand adding team-member matching for the free-textsearch_data_*tools, plus contract parity — is deferred to a follow-up PR stacked on #921 to keep this change independent and easy to review. Original-ID search coverage will land there too.Tests
New:
src/backend/src/tests/unit/test_mcp_tool_projections.py— 11 unit tests pinning the projection contract for products and contracts: team projection with roles, no-team case, malformed-member case (onlyusername), timestamp (de)serialization, andontosOriginalIdlookup for both the list-of-CustomProperty shape (products) and the dict shape (contracts).Test plan
get_data_producton an imported ODPS fixture via the MCP client — assertteam.members[*].roleand the four timestamps appear.get_data_products(bulk) with 2–3 ids — shape identical to single-get (both call_serialize_product).<fixture>?") — expect the imported team members with roles to appear in the answer.get_data_contract,get_data_contracts, agent query with ODCS fixture.original_idis populated for a product imported withadopt_ids=falseandNoneotherwise.This pull request and its description were written by Isaac.