Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Facilitate iteration in differential subworkflow #7115

Merged
merged 34 commits into from
Dec 11, 2024

Conversation

pinin4fjords
Copy link
Member

@pinin4fjords pinin4fjords commented Nov 28, 2024

Objective

To facilitate iterative plans proposed by community members for the differentialabundance pipeline.

Key Changes

  1. Iterative Processing

    • Introduce ch_inputs channel for iterative content
    • Contents: meta, abundance matrix, and analysis method, plus thresholds (since these might be dependent on method etc)
  2. Static Inputs

    • Contrasts and sample sheet treated as once-per-pipeline inputs
    • Combined with ch_inputs using Nextflow's combine operator
  3. Module Parameterization

    • DESeq2 and limma modules receive matrix-level meta
    • Enables custom parameters via closures in pipeline modules.conf

Implementation

  • Iterate through input matrices and parameter sets for all specified contrasts
  • Use combine operation to merge static and iterative inputs

This is quite like the differential subworkflow proposal in https://github.com/nf-core/differentialabundance/pull/288/files, except:

  • Single multiMap required
  • No duplication of filtering subworkflow import
  • Reduced need to muck about with meta combinations in the subworkflow
  • No assumptions about meta elements (method is part of the main channel content)

PR checklist

Closes #XXX

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the module conventions in the contribution docs
  • If necessary, include test data in your PR.
  • Remove all TODO statements.
  • Emit the versions.yml file.
  • Follow the naming conventions.
  • Follow the parameters requirements.
  • Follow the input/output options guidelines.
  • Add a resource label
  • Use BioConda and BioContainers if possible to fulfil software requirements.
  • Ensure that the test works with either Docker / Singularity. Conda CI tests can be quite flaky:
    • For modules:
      • nf-core modules test <MODULE> --profile docker
      • nf-core modules test <MODULE> --profile singularity
      • nf-core modules test <MODULE> --profile conda
    • For subworkflows:
      • nf-core subworkflows test <SUBWORKFLOW> --profile docker
      • nf-core subworkflows test <SUBWORKFLOW> --profile singularity
      • nf-core subworkflows test <SUBWORKFLOW> --profile conda

@pinin4fjords pinin4fjords changed the title Iterate differential Facilitate iteration in differential subworkflow Dec 6, 2024
@pinin4fjords
Copy link
Member Author

Just trying to track down the cryptic timezone issues in the CI...

@suzannejin suzannejin linked an issue Dec 10, 2024 that may be closed by this pull request
@suzannejin suzannejin added this pull request to the merge queue Dec 11, 2024
Merged via the queue into master with commit 784b3a3 Dec 11, 2024
27 checks passed
@suzannejin suzannejin deleted the iterate_differential branch December 11, 2024 16:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create 'differential' subworkflow
4 participants