Repository navigation
Test check_query_denominator; record measured harness coverage (#819) - #822
Merged
Merged
Conversation
test_scorer_calibration.sh never extracted check_query_denominator, so every run_truvari in the suite called it as an undefined command (rc 127) and pushed each stratum onto QUERY_FILTERED with nothing asserting it. Extract it, make the filter stub write TP-comp as truvari does, and test its three outcomes plus the QUERY_FILTERED wiring, with three mutations. Replace CLAUDE.md's stale rule-6 example with the measured state (25 of 29 sv_pipeline.sbatch functions exercised) and stop release-gates.yml from counting its ignored tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 7, 2026
Open
Open
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Steps 1 and 2 of the #819 testing audit. Test and docs only; no change to eidolon or to what the pipeline does on Delta.
Step 1:
#[ignore]tests (no change needed)release-gatesruns on every PR intodevelop. The log of run 37473380770 (#821) shows all 8 ignored tests executing and passing, each in 7–30 s: five realigned SV gates, the transition-matrix round trip, the golden-BAM NM check and the validate differential. Each one fails with a message if its tool is missing; none can skip silently. The workflow header said "six" and named only the gates and the differential. It now lists them without a count.Step 2: harness coverage of
sv_pipeline.sbatchAll 18 suites in
scripts/delta/tests/pass, and all 232 of their mutations are caught locally. CI already runs both modes and fails a suite that reports no mutation summary.Measured: 25 of 29 functions are exercised, counting functions reached through a tested caller (
run_truvari_both→run_truvari→comp_sample,check_denominator). This replaces CLAUDE.md's rule-6 example, which still said 2 of 14 and listedcheck_denominatoras untested.Untested:
score_caller,build_bnd_spans,convert_manta_inversions,index_and_align. These are reported on #819, not fixed here.The defect fixed
test_scorer_calibration.shnever extractedcheck_query_denominator. Everyrun_truvariin the suite therefore called an undefined command, and|| QUERY_FILTERED+=(…)flagged each stratum. Output went to/dev/nulland nothing assertedQUERY_FILTERED, so the suite reported 132 passed. Shown before the fix:check_query_denominator: command not found, thenQUERY_FILTERED=regression regression_allcalls.Fix:
filterstub writesTP-comp, as truvari does. Without it, every case took the "UNVERIFIED" branch.TP-comp/FPis reported UNVERIFIED and not flagged;--passonly (4 non-PASS);run_truvari_both, an all-non-PASS query flags exactly the PASS-only pass, and an all-PASS query flags nothing (must-not-fire).run_truvariignoring the result; counting no non-PASS records.bash -xshows nocommand not found. 142 passed, 0 mutations survived.Not verified
TP-compis derived the way truvari defines it, but it has not been compared against a real truvari summary.Refs #819.
🤖 Generated with Claude Code