Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bulk rnaseq multiqc per sample #67

Open
wants to merge 14 commits into
base: main
Choose a base branch
from

Conversation

AlaaALatif
Copy link
Member

key modifications to the pipeline:

  • MultiQC report per sample
    • in addition to a global multiQC report, there is now an individual report for each sample ID
    • reports now include additional metrics from SNP calling
  • Global CPU resource management
    • added executor.cpus parameter in config/base.config to be able to control maximum allowable number of CPU usage when running under local profile i.e. running the pipeline on a local machine or on a developmental (dev) node
    • added executor.queueSize parameter in config/base.config to be able to control maximum allowable number of jobs to run when running the pipeline on a cluster via slurm
    • added max_cpus_per_job parameter in config/parameters to be able to control maximum allowable number of CPU usage per job when running the pipeline on a cluster via slurm
    • the above two additions enable us to control the "ceiling" of number of CPUs the pipeline can use via slurm, preventing the risk of "hogging" C4 resources
  • Optional SNP calling
    • added call_snps parameter in config/parameters.config to be able to control whether or not SNP calling process steps need to be carried out. This was requested by the Genomics core and makes sense since these steps have considerably high computational requirements

@AlaaALatif AlaaALatif requested review from erflynn and dtm2451 March 7, 2024 19:45
@erflynn
Copy link
Collaborator

erflynn commented Mar 13, 2024

This is great!! Very excited to have the separated MQC outs and I think it's good to reduce the number of jobs for slurm. Such a bummer we can't easily set maximum cpus for slurm in nextflow, but I think what you have is a good call.
Only comment is that we may want to consider setting up a profile that takes more time but fewer cpus? Not part of this PR of course, but something for the future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants