Skip to content

Fix variant-effect/fine-mapping reproducibility bugs (T1/T2/T3) + reproduction audit#92

Merged
lucapinello merged 2 commits into
mainfrom
fix/2026-06-17-variant-scoring-reproducibility
Jun 17, 2026
Merged

Fix variant-effect/fine-mapping reproducibility bugs (T1/T2/T3) + reproduction audit#92
lucapinello merged 2 commits into
mainfrom
fix/2026-06-17-variant-scoring-reproducibility

Conversation

@lucapinello

Copy link
Copy Markdown
Contributor

Summary

Independent reproduction of the Chorus blog post (Analysis A: SORT1/rs12740374; Analysis B: rs9504151/CDYL) surfaced three correctness bugs in the variant-effect / fine-mapping path that broke "reproduce-from-prompt." This PR ships the validated, contained fixes plus the reproduction artifacts. Full evidence: audits/2026-06-16_blogpost_reproduction_report.md.

A separate, larger windowing bug (it requires rebuilding the per-track background CDFs) is documented but not fixed here — see audits/2026-06-17_windowing_normalization_P0.md.

Fixes

T2 — LD-proxy alleles were genome-mismatched (fine_map_causal_variant).
_extract_allele_pairs assigned each proxy (ref=SENTINEL_allele, alt=PROXY_allele) from Correlated_Alleles, so a proxy's "ref" was the sentinel's base — almost never the genome base at the proxy position (≈50 "Provided reference allele … does not match the genome" warnings for one locus; chorus then substituted the wrong ref → effects computed on wrong alleles → ranking meaningless).

  • chorus/utils/ld.py: each proxy now uses its own alleles (the Alleles column).
  • chorus/analysis/causal.py: prioritize_causal_variants orients ref→genome per variant before scoring.
  • Verified: re-run gives 0 mismatch warnings (was ~50); rs9504151 ranks borzoi_v1 with basic prediction and examples #1 (ChromBPNet IMR-90, effect −0.985).

T1 (fine-map path) — 1 bp scoring region collapsed fixed-input-oracle effects.
prioritize_causal_variants scored each variant on a 1 bp region; fixed-input oracles (e.g. ChromBPNet) map that into an N-padded output window, collapsing the effect ~4× (all proxies' |effect| ≤ 0.15). Now scores on the oracle's full input window centered on the variant (guarded for non-integer sequence_length). Verified: ChromBPNet effects restored (rs9504151 −0.985, matching a direct measurement).

Note: the same 1 bp pattern exists in _auto_region (4 MCP tools), discovery.py, batch_scoring.py, and build_backgrounds.py. Because the background CDFs were built with the collapsed window, fixing those requires a coordinated CDF rebuild — tracked as a P0 follow-up (audits/2026-06-17_windowing_normalization_P0.md), not in this PR.

T3 — AlphaGenome all-tracks scan OOM-killed the MCP server.
analyze_variant_multilayer / discover_variant with empty assay_ids on AlphaGenome expands to all 5,731 tracks; at 1 Mb the returned arrays (tens of GB) OOM-killed the whole server. chorus/oracles/alphagenome.py _predict now estimates the allocation and raises a clear ValueError before allocating (cap = 40 % of RAM, override CHORUS_AG_MAX_PREDICT_GB). Verified: blocks the 5,731-track@1Mb case, allows real runs (545- and 1,839-track scans pass).

Tests

  • tests/test_analysis.py: the old Correlated_Alleles "fanout" test asserted the buggy ref=SENTINEL contract; updated to assert the corrected own-alleles behavior. (Only one test depended on it.)
  • Full suite green after the fixes.

Docs / artifacts

  • audits/2026-06-16_blogpost_reproduction_report.md — full per-claim reproduction.
  • audits/2026-06-16_reproduction_notes.md — raw run numbers.
  • audits/2026-06-17_windowing_normalization_P0.md — the deferred windowing+CDF-rebuild bug.
  • audits/2026-006_chorus_blogpost.revised.md — reviewer-corrected article (number fixes + reproducibility recipes).

🤖 Generated with Claude Code

lp698 and others added 2 commits June 17, 2026 05:09
Independent reproduction of the Chorus blog post (SORT1/rs12740374 and
rs9504151/CDYL) surfaced three correctness bugs that broke reproduce-from-prompt:

- T2: fine_map_causal_variant scored LD proxies on the SENTINEL's allele as
  their ref (from Correlated_Alleles), almost never matching the genome at the
  proxy position -> wrong alleles -> meaningless ranking. Now each proxy uses
  its own alleles (chorus/utils/ld.py) and ref is oriented to the genome
  (chorus/analysis/causal.py). Verified: 0 genome-mismatch warnings (was ~50);
  rs9504151 ranks #1 (ChromBPNet IMR-90, -0.985).

- T1 (fine-map path): each variant was scored on a 1 bp region, which
  fixed-input oracles (e.g. ChromBPNet) map into an N-padded output window,
  collapsing the effect ~4x. Now scores on the oracle's full input window
  centered on the variant (guarded for non-integer sequence_length).

- T3: analyze_variant_multilayer/discover_variant with empty assay_ids on
  AlphaGenome expands to all 5731 tracks; at 1 Mb the result OOM-killed the
  MCP server. chorus/oracles/alphagenome.py now estimates the allocation and
  raises a clear ValueError before allocating (cap = 40% RAM, override
  CHORUS_AG_MAX_PREDICT_GB).

tests/test_analysis.py: the old Correlated_Alleles fanout test asserted the
buggy ref=SENTINEL contract; updated to the corrected own-alleles behavior.
Full suite: 407 passed, 5 skipped, 0 failed.

The broader windowing bug (same 1 bp pattern in _auto_region/discovery/
batch_scoring/build_backgrounds; affects all fixed-input oracles and requires a
per-track background-CDF rebuild) is documented as a P0 follow-up, NOT fixed
here: audits/2026-06-17_windowing_normalization_P0.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
AlphaGenome fine-map (lung-fibroblast tracks, T1+T2 fixes, 0 allele warnings)
ranks rs9504151 #1 of the 56-proxy credible set (composite 0.995, alt effect
-1.363) — confirming the draft's headline Analysis-B claim. Both oracles now
agree: AlphaGenome #1 and ChromBPNet IMR-90 #1. Updated the revised article and
the reproduction report.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@lucapinello lucapinello merged commit 4ad7be7 into main Jun 17, 2026
1 check failed
@lucapinello lucapinello deleted the fix/2026-06-17-variant-scoring-reproducibility branch June 17, 2026 13:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant