Repository navigation
fix(qa-mc2dp): add disease_type/primary_site to aggMDS GDC adapter mapping - #575
Merged
Merged
Conversation
…pping GDCAdapter builds each study description from the mapped disease_type and primary_site fields, so the mapping added in #573 crashed with KeyError: 'disease_type' and MDS left the existing indexes (VPODC data) in place. With the two fields mapped the adapter returns the 2 APOLLO projects, which lets populate.py swap the indexes and drop the old data. .secrets.baseline regenerated by the detect-secrets pre-commit hook. Claude-Session: https://claude.ai/code/session_019WtPvAbe8JKDofTACKqa7D
jawadqur
added a commit
that referenced
this pull request
Nov 18, 2026
- Resolve the conflict with #575: keep _unique_id=path:submitter_id and the disease_type/primary_site mappings GDCAdapter requires. - subject_type_of_exposures: map exposures[*].tobacco_smoking_status (as prod mc2dp does). "path:exposures" yields a list of dicts, which the string schema type can't convert, crashing the job with "TypeError: sequence item 0: expected str instance, dict found". - Append ",id" to the cases fields list. GDCAdapter appends "?expand=summary&from=..&size=.." to mds_url, which gets glued onto the last field, so project.project_id was never returned (data_source_project was null for every record). - Restore the closing brace missing from .secrets.baseline on master, which made the detect-secrets hook fail. Tested locally with MDS master's GDCAdapter against the live GDC API: 232 subjects (APOLLO-LUAD 87, APOLLO-OV 70, APOLLO-BRCA-1 68, VAREPOP-APOLLO 7). Claude-Session: https://claude.ai/code/session_019WtPvAbe8JKDofTACKqa7D
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #573. The GDC adapter added there crashes the aggMDS job:
GDCAdapteralways readsdisease_typeandprimary_sitefrom the mapped fields, and the mapping didn't define them. So nothing was written, and the old indexes (including the VPODC data we're trying to remove from QA) stayed in place.This adds the two mappings. I ran MDS master's
GDCAdapterandparse_configlocally against this config and the live GDC API:KeyError: 'disease_type'VAREPOP-APOLLO,APOLLO-LUAD), sopopulate.pygets past its "0 records → keep existing indexes" guard and swaps the indexes..secrets.baselinewas regenerated by the repo's detect-secrets hook (v1.5.0). It drops 3 stale entries for this file that the scanner no longer flags.https://claude.ai/code/session_019WtPvAbe8JKDofTACKqa7D