Add residual design-effect correction and aggregation calibration parameters - #146
Merged
Conversation
mschwoer
reviewed
Aug 4, 2026
Comment on lines
69
to
+71
| df_c1_normed, df_c2_normed = aqnorm.normalize_if_specified(df_c1 = df_c1, df_c2 = df_c2, c1_samples = c1_samples, c2_samples = c2_samples, normalize_within_conds = runconfig.normalize, normalize_between_conds = runconfig.normalize, | ||
| runtime_plots = runconfig.runtime_plots, protein_subset_for_normalization_file=runconfig.protein_subset_for_normalization_file, pep2prot = pep2prot)#, "./test_data/normed_intensities.tsv") | ||
| runtime_plots = runconfig.runtime_plots, protein_subset_for_normalization_file=runconfig.protein_subset_for_normalization_file, pep2prot = pep2prot, | ||
| median_normalization = getattr(runconfig, 'median_normalization', False))#, "./test_data/normed_intensities.tsv") |
Contributor
There was a problem hiding this comment.
can you please simplify this by splitting into several statements?
Contributor
There was a problem hiding this comment.
nevermind, now I got it..
ping me once it is a good time to format the codebase ;-)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Corrects the between-peptide correlation that residual-decorrelation pruning cannot remove, by measuring the excess sibling correlation over the cross-parent shuffle null and applying it as a Stouffer design effect (
deff=1+(n-1)*rho).It is additionally gated on that level's pre-pruning distance exceeding the tolerance, so when peptides are no more correlated than the shuffle null the gate stays closed,
rhoremains0.0and aggregation is bit-identical to before. A single level-wide excessrhois applied per level, clipped at zero once on the level mean rather than per parent, since per-parent means are noisy and clipping each would rectify that noise into a positive bias. Below a total-sample threshold the source switches to raw pre-pruning correlations, because at low replicate counts the survivorrhoreads ~0 while the correlation still leaks into the Stouffer sum.Also adds the calibration parameters this work required, each as its own commit:
median_normalization— take the median of the between-condition fold-change distribution as the shift instead of choosing between its median and its mode.max_peptides_per_protein— replaces the hard-coded 31-peptide cap with a configurable one.residual_decorrelation_cutoff_grid— the default grid now extends to -1.0; the negative part is reached only when no cutoff meets the tolerance, in which case the tightest prunes down tomin_keep.residual_decorr_corr_mode/_cap— bounds how many samples the sibling-correlation estimate may use, so pruning aggressiveness does not grow with sample count.median_on_collapse— when pruning collapses a parent to a single surviving child, aggregate via the median of all eligible children rather than reporting the lone survivor. Sincemedian_zdoes not scale bysqrt(n), no design effect applies on this path, and single-peptide parents are unaffected at anyrho.runtime_plotsis set.Behavior changes at default settings
max_peptides_per_proteindefaults to no cap; the previous behavior was effectively a cap of 31.median_on_collapseandresidual_deff_correctiondefault to on.Testing
tests/unit_tests: 185 passed, 1 skipped. Each commit parses on its own, so the history is bisectable.