Align scdownstream with sc-best-practices defaults and backlog#303
Open
nictru wants to merge 25 commits into
Open
Align scdownstream with sc-best-practices defaults and backlog#303nictru wants to merge 25 commits into
nictru wants to merge 25 commits into
Conversation
Update samplesheet schema defaults (MAD, mito cap, min_cells), default doublet detection to scDblFinder, and add --doublet_removal for optional filtering. Centralise QC thresholds in schema_input.json and make SCANPY_FILTER skip unset thresholds while always computing QC metrics.
Swap default ambient correction to scAR and doublet detection to Scrublet when running with -profile python_only alongside a container profile.
Introduce scry/deviance module and --feature_selection (hvgs|deviance|none), defaulting to deviance per sc-best-practices, with python_only and tests pinned to HVGs and integration_hvgs renamed to integration_n_features.
… subworkflows. Unify rank-genes-groups, pseudobulk, and edgePython sc DE under de_methods; extract RANK_GENES_GROUPS, orchestrate via DIFFERENTIAL_EXPRESSION, remove skip_rankgenesgroups, and ensure CyteType always runs on Wilcoxon markers.
Unify per-sample donor_col to obs donor during QC, infer pseudobulking from DE methods, and expand MultiQC marker-gene and QC reporting with updated docs and tests.
Introduce log1p, scran, and Pearson residual modules with a NORMALIZATION subworkflow, normalization_methods and transformed_layer parameters, and native sparse layer fragments merged via ch_layers at finalize.
Standardise per-cell and cluster annotation column propagation via manifests, add cluster annotation orchestration, and ensure filtered marker genes feed reports and downstream consumers. Generated by GPT-5.5.
Set Leiden to igraph, expose neighbors_n_pcs, add optional t-SNE, and wire Scanorama integration with Symphony-style preprocessing and container configs. Generated by Composer
|
Default to scran normalisation with transformed_layer=scran, override in python_only to log1p/log1p_norm, and document which integrations consume the layer versus raw counts or internal normalisation.
Add pearson_residuals_hvgs with counts_layer-aware HVG selection on raw counts, remove the pearson_residuals normalisation layer, and update schema, docs, and nf-tests accordingly.
Layer-aware integrations now always read the precomputed log1p or scran layer with explicit log_normalize=false, and the shifted-log AnnData layer is renamed to log1p for alignment with the method name.
INTEGRATE now delegates HVG, deviance, and Pearson-residual gene filtering plus empty-cell removal to FEATURE_SELECTION, matching the NORMALIZATION split.
Regenerate snapshots for modules whose outputs changed under the local apptainer/daisybio test profile. Generated by Composer
Use `container` instead of `conda` in lock-file configs so CI matches the format produced by nf-core/tools 4.0.2.
Regenerate snapshots for subworkflows whose outputs changed under the local apptainer/daisybio test profile. Generated by Composer
Build a dedicated SCRAN_NORMALIZATION container, align combine/sub_integrate tests with the single normalization_method input, and harden cell-cycle, rank-genes, and deviance outputs.
…ce paths. Harden module templates for gene symbol handling, size factors, and empty DE results, align quality-control assertions with filtered outputs, and refresh snapshots. Generated by Composer
Correct the swapped PER_GROUP inputs in scdownstream, switch SoupX-sensitive pipeline tests to decontX, remove the redundant build_minimal test, and update passing pipeline snapshots.
Normalise complete parents once, keep layers in H5ADs, subset intersection genes for integration via ADATA_SUBSETVAR, and stop forwarding integration feature-selection var fragments to FINALIZE so union parents finalize without axis mismatches.
Use structural snapshots for the normalization subworkflow test and regenerate pipeline snapshots after the COMBINE/FINALIZE output changes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR aligns nf-core/scdownstream with the sc-best-practices workflow and closes the remaining pipeline backlog items tracked in the publication concordance doc.
min_cells=20, SoupX ambient correction, scDblFinder with annotate-only doubletslog1p,scran, and Pearson residual layers vianormalization_methods/transformed_layerde_methodsframework with pseudobulk PyDESeq2/edgePython auto-triggering; filtered Wilcoxon cluster markers in reportsannotation:naming, manifest-driven cluster aggregation, and optional CyteType integrationflavor=igraph; newneighbors_n_pcsparameter; optional--tsneembeddings; Scanorama integration with Symphony-style preprocessingAlso adds a
python_onlyprofile (scAR + Scrublet + HVGs) and updates usage, output, and reproducibility documentation.Generated by Composer
Test plan
nf-core pipelines lintnf-test teston affected modules and subworkflows (cluster, integrate, scanorama, tsne, neighbors, leiden)nextflow run . -profile test,docker --outdir <OUTDIR>