Fix variant-effect/fine-mapping reproducibility bugs (T1/T2/T3) + reproduction audit#92
Merged
lucapinello merged 2 commits intoJun 17, 2026
Conversation
Independent reproduction of the Chorus blog post (SORT1/rs12740374 and rs9504151/CDYL) surfaced three correctness bugs that broke reproduce-from-prompt: - T2: fine_map_causal_variant scored LD proxies on the SENTINEL's allele as their ref (from Correlated_Alleles), almost never matching the genome at the proxy position -> wrong alleles -> meaningless ranking. Now each proxy uses its own alleles (chorus/utils/ld.py) and ref is oriented to the genome (chorus/analysis/causal.py). Verified: 0 genome-mismatch warnings (was ~50); rs9504151 ranks #1 (ChromBPNet IMR-90, -0.985). - T1 (fine-map path): each variant was scored on a 1 bp region, which fixed-input oracles (e.g. ChromBPNet) map into an N-padded output window, collapsing the effect ~4x. Now scores on the oracle's full input window centered on the variant (guarded for non-integer sequence_length). - T3: analyze_variant_multilayer/discover_variant with empty assay_ids on AlphaGenome expands to all 5731 tracks; at 1 Mb the result OOM-killed the MCP server. chorus/oracles/alphagenome.py now estimates the allocation and raises a clear ValueError before allocating (cap = 40% RAM, override CHORUS_AG_MAX_PREDICT_GB). tests/test_analysis.py: the old Correlated_Alleles fanout test asserted the buggy ref=SENTINEL contract; updated to the corrected own-alleles behavior. Full suite: 407 passed, 5 skipped, 0 failed. The broader windowing bug (same 1 bp pattern in _auto_region/discovery/ batch_scoring/build_backgrounds; affects all fixed-input oracles and requires a per-track background-CDF rebuild) is documented as a P0 follow-up, NOT fixed here: audits/2026-06-17_windowing_normalization_P0.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
AlphaGenome fine-map (lung-fibroblast tracks, T1+T2 fixes, 0 allele warnings) ranks rs9504151 #1 of the 56-proxy credible set (composite 0.995, alt effect -1.363) — confirming the draft's headline Analysis-B claim. Both oracles now agree: AlphaGenome #1 and ChromBPNet IMR-90 #1. Updated the revised article and the reproduction report. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Independent reproduction of the Chorus blog post (Analysis A: SORT1/rs12740374; Analysis B: rs9504151/CDYL) surfaced three correctness bugs in the variant-effect / fine-mapping path that broke "reproduce-from-prompt." This PR ships the validated, contained fixes plus the reproduction artifacts. Full evidence:
audits/2026-06-16_blogpost_reproduction_report.md.A separate, larger windowing bug (it requires rebuilding the per-track background CDFs) is documented but not fixed here — see
audits/2026-06-17_windowing_normalization_P0.md.Fixes
T2 — LD-proxy alleles were genome-mismatched (
fine_map_causal_variant)._extract_allele_pairsassigned each proxy(ref=SENTINEL_allele, alt=PROXY_allele)fromCorrelated_Alleles, so a proxy's "ref" was the sentinel's base — almost never the genome base at the proxy position (≈50 "Provided reference allele … does not match the genome" warnings for one locus; chorus then substituted the wrong ref → effects computed on wrong alleles → ranking meaningless).chorus/utils/ld.py: each proxy now uses its own alleles (theAllelescolumn).chorus/analysis/causal.py:prioritize_causal_variantsorients ref→genome per variant before scoring.T1 (fine-map path) — 1 bp scoring region collapsed fixed-input-oracle effects.
prioritize_causal_variantsscored each variant on a 1 bp region; fixed-input oracles (e.g. ChromBPNet) map that into an N-padded output window, collapsing the effect ~4× (all proxies' |effect| ≤ 0.15). Now scores on the oracle's full input window centered on the variant (guarded for non-integersequence_length). Verified: ChromBPNet effects restored (rs9504151 −0.985, matching a direct measurement).T3 — AlphaGenome all-tracks scan OOM-killed the MCP server.
analyze_variant_multilayer/discover_variantwith emptyassay_idson AlphaGenome expands to all 5,731 tracks; at 1 Mb the returned arrays (tens of GB) OOM-killed the whole server.chorus/oracles/alphagenome.py_predictnow estimates the allocation and raises a clearValueErrorbefore allocating (cap = 40 % of RAM, overrideCHORUS_AG_MAX_PREDICT_GB). Verified: blocks the 5,731-track@1Mb case, allows real runs (545- and 1,839-track scans pass).Tests
tests/test_analysis.py: the oldCorrelated_Alleles"fanout" test asserted the buggyref=SENTINELcontract; updated to assert the corrected own-alleles behavior. (Only one test depended on it.)Docs / artifacts
audits/2026-06-16_blogpost_reproduction_report.md— full per-claim reproduction.audits/2026-06-16_reproduction_notes.md— raw run numbers.audits/2026-06-17_windowing_normalization_P0.md— the deferred windowing+CDF-rebuild bug.audits/2026-006_chorus_blogpost.revised.md— reviewer-corrected article (number fixes + reproducibility recipes).🤖 Generated with Claude Code