Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stubs and test for rnaseq modules - part IV #5939

Merged
merged 24 commits into from
Jul 12, 2024

Conversation

maxulysse
Copy link
Member

Improve/fix stub, stub tests + single snapshot + snapshot versions:

  • untar

PR checklist

Closes #XXX

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the module conventions in the contribution docs
  • If necessary, include test data in your PR.
  • Remove all TODO statements.
  • Emit the versions.yml file.
  • Follow the naming conventions.
  • Follow the parameters requirements.
  • Follow the input/output options guidelines.
  • Add a resource label
  • Use BioConda and BioContainers if possible to fulfil software requirements.
  • Ensure that the test works with either Docker / Singularity. Conda CI tests can be quite flaky:
    • For modules:
      • nf-core modules test <MODULE> --profile docker
      • nf-core modules test <MODULE> --profile singularity
      • nf-core modules test <MODULE> --profile conda
    • For subworkflows:
      • nf-core subworkflows test <SUBWORKFLOW> --profile docker
      • nf-core subworkflows test <SUBWORKFLOW> --profile singularity
      • nf-core subworkflows test <SUBWORKFLOW> --profile conda

@maxulysse maxulysse requested a review from jasmezz as a code owner July 10, 2024 11:56
@maxulysse maxulysse requested a review from rpetit3 as a code owner July 11, 2024 10:35
@maxulysse
Copy link
Member Author

for untar and all other containers using ubuntu: #5968

@maxulysse
Copy link
Member Author

Issues with extracting minigut_cf.tar.gz on docker_self_hosted due to Cannot change ownership to uid 253902119, gid 50: Invalid argument:

  • centrifuge/centrifuge
  • centrifuge/kreport

Issues with extracting data.tar.gz on singularity/docker_self_hosted, due to missing lbzip2 in container:

  • rgi/cardannotation
  • rgi/main
  • hamronization/rgi

Unknown issue on docker_self_hosted for:

  • checkv/updatedatabase

@@ -8,7 +8,7 @@
"id": "contigs",
"single_end": false
},
"contigs.STUB.train:md5,d41d8cd98f00b204e9800998ecf8427e"
"contigs..des.train:md5,d41d8cd98f00b204e9800998ecf8427e"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this have two dots now?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just replace the INDEX_NAME=STUB in the stub by the same command from the script:
INDEX_NAME=\$(basename \$(ls $index/*.des) .des)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the stub does not receive a .des file, ls $index/*.des returns nothing and the basename command becomes basename .des, which returns .des.

@maxulysse maxulysse enabled auto-merge July 12, 2024 13:37
@maxulysse maxulysse disabled auto-merge July 12, 2024 13:37
@maxulysse maxulysse merged commit 4e5f468 into nf-core:master Jul 12, 2024
282 of 291 checks passed
@maxulysse
Copy link
Member Author

Self-merging

@maxulysse maxulysse deleted the stubs_rnaseq_viii branch July 12, 2024 13:38
mirpedrol added a commit that referenced this pull request Sep 20, 2024
* Update Abricate to include (missing) additional optional input channel and add stub (#5849)

* Add missing optional input channel to abricate

* Update abricate module to include missing optional input path

* Update modules/nf-core/abricate/run/tests/main.nf.test

Co-authored-by: Carson J Miller <[email protected]>

* Fix linting

* Update modules/nf-core/abricate/run/main.nf

* Get downstream module test

---------

Co-authored-by: Carson J Miller <[email protected]>

* Fix custom/catadditionalfasta (#5851)

* Update environment.yml

* Update main.nf

* update to samtools 1.20 (#5852)

* update to samtools 1.20

* keep samtools 1.19.2 for now

* update snapshot

* [automated] Fix linting with Prettier

* keep 1.20 afterall

* update snapshot

---------

Co-authored-by: nf-core-bot <[email protected]>

* Update input params for bwameme (#5853)

* update recipe

* update rthreads

* update tests

* Update meta.yml

* chore(deps): update pre-commit hook astral-sh/ruff-pre-commit to v0.4.10 (#5857)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* `--output-unselected` option in `samtools view` (#2674)

* add unoutput option

* trying to fix emit

* CI fixed, tests still incorrect

* Fixed the checksums

* Fixed the syntax

* Call this "unselected" rather than "unoutput"

* Alignment rocks

* bugfix: the variable needs to be visible by the output declaration

* bugfix: this was not the regular file name

* Updated the test case

* fixup! Alignment rocks

* Updated the meta.yml

---------

Co-authored-by: Matthieu Muffato <[email protected]>

* stubs and test for rnaseq modules - part I (#5854)

* stubs and test for bbmap_bbsplit

* add test for index

* add stubs for custom/catadditionalfasta

* add stubs for fq/subsample

* add stubs for hisat2/extractsplicesites

* update fastq_align_hisat2 snapshot

* add stubs for hisat2/build

* add stubs for hisat2/align

* more stubs bbmap/bbsplit

* add stubs for preseq/lcextrap

* more stubs for fq/subsample

* proper stubs for hisat2/align

* remove old snaps

* Remove CUTUPFASTA from pytests

* Revert "Remove CUTUPFASTA from pytests"

This reverts commit 0fa8c1b.

* Bump concoct/cutupfasta version, add stub, add nf-test (#5864)

* Bump concoct/cutupfasta version, add stub, add nf-test

* Fix linting

* Adding meta to multivcfanalyzer, updating test to nf-test (#5773)

* Adding meta to multivcfanalyzer, updating test to nf-test

* Apply suggestions from code review

Co-authored-by: James A. Fellows Yates <[email protected]>

* fix meta.yml

* Add sort to vcfs

* Add tab to tags

* add checks for files due to failing md5 sums across docker,conda,singularity

---------

Co-authored-by: James A. Fellows Yates <[email protected]>
Co-authored-by: Simon Pearce <[email protected]>

* Concoct/concoctcoveragetable modules updates (#5868)

* Bump concoct/cutupfasta version, add stub, add nf-test

* Bump conda build, add stub and nf-test for CONCOT/concoctcoveragetable

* Remove pytest entry

---------

Co-authored-by: Aida Andrades Valtueña <[email protected]>

* Added stub logic and test to fastq_fastqc_umitools_fastp (#5858)

* Added stub logic and test to fastq_fastqc_umitools_fastp

* Now emitting trimmed reads

* Adding input value-channel for controlling spring-decompress cmd and output (#5850)

* Enable single fastq.gz-output from spring/decompress by setting meta.single_end or meta.one_strand

* Controlling output with value-channel write_one_fastq_gz

* Update modules/nf-core/spring/decompress/meta.yml

Co-authored-by: Maxime U Garcia <[email protected]>

* nf-test for spring/decompress

* Trying to please linter

* Trying to please linter

---------

Co-authored-by: Simon Pearce <[email protected]>
Co-authored-by: Maxime U Garcia <[email protected]>

* Update stranger  (#5717)

* everything but snapshot

* fix gz snap

* update testdata paths

* Update main.nf.test

* update test

---------

Co-authored-by: Simon Pearce <[email protected]>

* mapAD: fix `versions.yml` emission (#5874)

mapAD: fix `version.yml` emission

* add stub and stub test to chromap, change test names (#5872)

* add stub and stub test to chromap, change test names

* add stub and stub test to chromap, change test names

* change setup

* move setup out of test

* move

* produce stub files (#5875)

* chore(deps): update pre-commit hook python-jsonschema/check-jsonschema to v0.28.6 (#5878)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* update peddy tests (#5880)

update peddy

* Update main.nf (#5870)

* Update main.nf 

VEP html report is missing when publishing results because by default vep html file is `_summary.html` instead of `.summary.html`. This changes just adjust the module to the vep defaults to avoid the issue.

* Updated html pattern in VEP to be more generic

* bump MultiQC version (#5884)

bump MQC

* Bump shinyngs modules (#5881)

* Bump shinyngs modules

* Bump versions for snapshot

* Update CONCOCT module(s): conda build bump, stub, and nf-test  (#5869)

* Version update, stub, nf-test for concoct/concoct

* Bump version, add nf-test, stub to concoct/mergecutupclustering

* nf-test, stub, and version conda build version bump for concoct/extrafastabins

* Remove a superfluous config and specify a used one

* Update modules/nf-core/concoct/extractfastabins/main.nf

Co-authored-by: Simon Pearce <[email protected]>

* Remove superfluous config file

* Clean up code

* Repair all stub tests that were empty

---------

Co-authored-by: Simon Pearce <[email protected]>

* `fasta_binning_concoct` subworkflow: do not fail on join (#5824)

* fix: do not fail on mismatch for samples not binned with concoct

* Migrate to pytest

* Improved Harshil alignment

---------

Co-authored-by: James A. Fellows Yates <[email protected]>

* new module: datavzrd (#5883)

* Generated structure for new module

* Implemented module

* Added semi-functional tests

* Cleaned tests - test don't work yet

* fix main, start nf-test

* add new test data

* fix linting

* try to assert folders

* fix tests

* change output to prefix

* change to prefix

* update to prefix

---------

Co-authored-by: vickylaram <[email protected]>

* samtools/reheader: Fix bug causing output name to be incorrectly evaluated (#5887)

Fix bug causing output name to be incorrectly evaluated

* nf-test for snpsites (#5284)

* nf-test for snpsites

* remove old pytest

* Delete modules/nf-core/snpsites/tests/nextflow.config

* Update modules/nf-core/snpsites/tests/main.nf.test

Co-authored-by: Joon Klaps <[email protected]>

* Update assert

Co-authored-by: Joon Klaps <[email protected]>

* renamed test

Co-authored-by: Joon Klaps <[email protected]>

* reverted the input

* updated input...again

Co-authored-by: Joon Klaps <[email protected]>

* fixed typo

Co-authored-by: Joon Klaps <[email protected]>

* Regenerate snapshot and fix linting

---------

Co-authored-by: Joon Klaps <[email protected]>
Co-authored-by: Simon Pearce <[email protected]>

* Added prefix to gunzip and while keeping the default name as `archive.toString() - '.gz'`  (#5879)

* Added prefix to gunzip

* Fixed linting issues in fargene and leviosam2/lift

* Removed unstable liftoff/conda

* add dedicated output channel for trimmomatic stderr log (needed for multiqc) (#5501)

* created new output channel capturing stderr of trimmomatic

* updated output channel for trimmomatic

* Add stub test and update snapshots

* Add stub and update tests

* Fix linting and add tee

---------

Co-authored-by: John Palmer <[email protected]>
Co-authored-by: Simon Pearce <[email protected]>
Co-authored-by: James A. Fellows Yates <[email protected]>

* `ariba/getref` port to nf-test, add meta to tuple and stub (#5877)

* port to nf-test, add meta to tuple and stub

* modify meta and fix tags

* change from each to tuple

* Pass annotation files to last-dotplot (#5885)

* Pass annotation files to last-dotplot

The annotation of the target (plotted horizontally) genome is passed as
a new channel.

The annotation of the query (plotted vertically) genome is passed as a
new component of the first channel.

This is because this module typically loops on a list of alignments of
various query genomes (from samplesheet) to a single target genome.

The annotation files are called `annot_a` and `annot_b` because they are
passed to the `-a` and `-b` arguments of `last-dotplot`.

* Put arguments on separate lines.

* Add meta map and reorder channels.

* Test with dummy annotation files.

Thanks @SPPearce for the hint.

* Add meta to bowtie build (#3848)

* Add meta to bowtie build

* Swap fastq ngscheckmate tests

* Swap to nf-test

* Remove spurious changes to bowtie2

* Actually revert bowtie2

* Add the rest of bowtie2

* Again

* Revert bowtie2 changes

* Swap to meta.id as prefix

* Swap bowtie align to nftest

* Update tests

* Use nft-bam

* Update modules/nf-core/bowtie/build/tests/main.nf.test

Co-authored-by: Mahesh Binzer-Panchal <[email protected]>

* Swap to testdata path

* Update modules/nf-core/ngscheckmate/patterngenerator/main.nf

* Swap index finding to $

* Fix vafncm test

* Update meta

* Remove bam header md5sum

* Update fastq test

* Update paired bam test

* Delete modules/nf-core/bowtie/align/tests/nextflow.config

---------

Co-authored-by: Mahesh Binzer-Panchal <[email protected]>

* nf-test for pilon (#5293)

* nf-test for pilon

* linting for license

* Add stub and extra test

* Swap to testdata_base_path

* Add missing END_VERSIONS and alignment

* Add optional outputs to stub

---------

Co-authored-by: Simon Pearce <[email protected]>

* Bump phantompeakqualtools 1.2.2 (#5889)

* Add stub section

* Update yml

* Add nf-test

* Get rid of pytest

* Make lint happy

* Update tests

* Update tests to make lint happy

* update tests

* Make lint happy

* Using the same version as CI of nf-test locally might be a good idea

* Use wave images

* Make lint happy

* Simplify tests

---------

Co-authored-by: Simon Pearce <[email protected]>

* Added variance filtering to custom/matrixfilter (#5358)

* Added variant filtering to matrix filter

* Corrected indentation

* Simplified function in single conditional block

* fixed line indentation

* Fixed whitespace

* Fixed logic

* line wrapping change

* added whitespace around comments

* [automated] Fix linting with Prettier

* remove trailing whitespace

* Update matrixfilter.R

* Remove whitespace

---------

Co-authored-by: Jonathan Manning <[email protected]>
Co-authored-by: nf-core-bot <[email protected]>
Co-authored-by: Jonathan Manning <[email protected]>
Co-authored-by: Simon Pearce <[email protected]>

* Update md5sum module to be able to take multiple files (#3610)

* Fix version string spurious text in singularity tests

* Add config for subworkflow

* Update md5sum module to work with multiple files

* linting

* Fix stub

* Add option to generate single file

* Update modules/nf-core/md5sum/main.nf

Co-authored-by: Matthieu Muffato <[email protected]>

* Simplify versions processing

---------

Co-authored-by: Matthieu Muffato <[email protected]>

* Add share to find path (#5899)

* build: Bump deepvariant to 1.6.1 (#5801)

* build: Bump deepvariant to 1.6.1

* interim dir not allowed to be a dot in 1.6

* Update modules/nf-core/deepvariant/main.nf

Co-authored-by: Maxime U Garcia <[email protected]>

* hack: Hard code version

google/deepvariant#830

---------

Co-authored-by: Felix Lenner <[email protected]>
Co-authored-by: Felix Lenner <[email protected]>
Co-authored-by: Maxime U Garcia <[email protected]>

* bump somalier/extract and somalier/relate version (#5894)

* bump version

* nf-test migratino

* revert ancestry

* fix linting

* Update snap

* fix unicode space

---------

Co-authored-by: Simon Pearce <[email protected]>

* Added evigene/tr2aacds (#5898)

* Added evigene/tr2aacds

* Sorted file list and added activation script

* Fixed issues raised by feedback

* correct president stub (#5904)

* correct president stub

* Large update to module

* Update prefix and meta

---------

Co-authored-by: Simon Pearce <[email protected]>

* Fix EClint errors (#5907)

Fix EClint

* New module: nanoq (#5896)

* Add the new module nanoq

* Update modules/nf-core/nanoq/main.nf

Co-authored-by: Simon Pearce <[email protected]>

* Update modules/nf-core/nanoq/main.nf

Co-authored-by: Simon Pearce <[email protected]>

* define the output_format

* Update modules/nf-core/nanoq/main.nf

Co-authored-by: Simon Pearce <[email protected]>

* Update modules/nf-core/nanoq/main.nf

Co-authored-by: Simon Pearce <[email protected]>

* Update modules/nf-core/nanoq/main.nf

Co-authored-by: Simon Pearce <[email protected]>

* update the stub run with gzip

* Update modules/nf-core/nanoq/main.nf

Co-authored-by: Simon Pearce <[email protected]>

* Update modules/nf-core/nanoq/main.nf

Co-authored-by: Simon Pearce <[email protected]>

---------

Co-authored-by: Simon Pearce <[email protected]>

* Foldseek: createdb and easysearch modules version bump + stub nf-tests added (#5906)

* foldseek/createdb version bump, added stub nf-test

* foldseek/easysearch version bump, stub nf-test added

* test.snap for createdb added

* sourmash/compare stub, nf-test, snapshot (#4665)

* sourmash/compare stub, nf-test, snapshot

* Add sort

* Add a sort

* Update snap

* Try sort inside the module

* Add meta to output and change formatting on test

---------

Co-authored-by: Simon Pearce <[email protected]>

* adding chewbbaca/createschema module (#4512)

* adding chewbbaca/createschema module

* updated for handling a mix of uncompressed and compressed files

* update the formatting

* update the formatting

* updated module to emit meta

* updated test.yml

* update pytest

* clean the meta.yml

* clean the meta.yml

* clean the meta.yml

* replaced conda package with environment.yml file

* py-test to nf-test migration

* add stub and update tests, snapshot

* updated stub

* update stub test

* update stub test

* bump the version

* update

* updating container version

* updated test snapshot

* updated nf-test snapshot

* added libmamba to environment file

* Fix tests and linting

* Update modules/nf-core/chewbbaca/createschema/main.nf

Co-authored-by: Jose Espinosa-Carrasco <[email protected]>

* Apply suggestions from code review

Co-authored-by: Jose Espinosa-Carrasco <[email protected]>

---------

Co-authored-by: Gisela Gabernet <[email protected]>
Co-authored-by: Sateesh <[email protected]>
Co-authored-by: Simon Pearce <[email protected]>
Co-authored-by: Jose Espinosa-Carrasco <[email protected]>

* stubs and test for rnaseq modules - part II (#5861)

* add stubs for rsem

* add stubs for rseqc

* add stubs for subread/featurecounts

* add stubs for trimgalore

* add stubs for ucsc/bedclip

* add stubs and more tests for umitools/extract

* update snap

* update snapshot for umitools/extract

* fix umitools/extract tests

* Apply suggestions from code review

Co-authored-by: Simon Pearce <[email protected]>

---------

Co-authored-by: Simon Pearce <[email protected]>

* affy/justrma add stub, nf-test and snapshot (#4651)

* affy/justrma add stub, nf-test and snapshot

* Update snapshot and environment

* Update snap

* Update modules/nf-core/affy/justrma/main.nf

Co-authored-by: James A. Fellows Yates <[email protected]>

* Swap test data file paths, fix config, update snaps and stub

---------

Co-authored-by: Famke Bäuerle <[email protected]>
Co-authored-by: Simon Pearce <[email protected]>
Co-authored-by: James A. Fellows Yates <[email protected]>

* Bump fgbio versions and swap to nftest (#5624)

* Convert callduplex and callmolecular to nftest and update version

* Convert zipperbams

* Convert groupreadsbyumi

* Convert sortbam and add stubs

* Convert filterconsensusreads

* Update snapshot

* Update modules/nf-core/fgbio/callmolecularconsensusreads/main.nf

* Add collision checks, swap test data paths

* Fix conda version and linting

* Apply suggestions from code review

Co-authored-by: Nils Homer <[email protected]>

* Update modules/nf-core/fgbio/callmolecularconsensusreads/meta.yml

Co-authored-by: Nils Homer <[email protected]>

* Update duplex snap

---------

Co-authored-by: Nils Homer <[email protected]>
Co-authored-by: Maxime U Garcia <[email protected]>

* Added pbtk/pbindex (#5901)

* Added pbtk/pbindex

* Removed prefix

* Added agat/spfilterfeaturefromkilllist (#5771)

* Added agat/spfilterfeaturefromkilllist

* Added AGAT config

* Included test with agat config

* bump stranger to 0.9.1 (#5910)

bump stranger

* Update nft-bam to v0.3.0 (#5912)

* Add stubs for rnaseq subworkflows - part I (#5911)

* fix stub and add tests + single snapshot

* add stub

* fix stub

* snapshot versions + better snap for stub

* add versions to snapshot + add tests for stub

* Add stub tests + single snapshot

* Add stub tests + single snapshot

* maxime sorted

* Add stub tests + single snapshot

* Add stub tests

* Add stub tests + single snapshot

* code polish + update snap

* update snapshot using nft-bam

* Add stubs for rnaseq subworkflows - part II (#5913)

* Add stub tests + single snapshot

* Add stub tests + single snapshot

* fix stub and update tests for kallisto modules

* Add stub tests + single snapshot

* Add stub tests + single snapshot

* bcftools isec input can be bcf or bcf.gz as well as vcf.gz (#5685)

* bcftools isec input can be bcf or bcf.gz as well as vcf.gz

* Add stub and stub test

* Update tests

* Update meta.yml

* Add stub to name of stub test

* Update meta

* Update tests

Changes:
- exclude .tbi files from snapshot due to failing conda tests
- update main.nf.test.snap
- specificy path type of the  output

* Update tests again

Changes:
- include versions
- sort file name list for consistent order
- update snapshot

---------

Co-authored-by: Simon Pearce <[email protected]>
Co-authored-by: Theresa Störiko <[email protected]>
Co-authored-by: Theresa Störiko <[email protected]>

* Add stubs for rnaseq subworkflows - part III (#5921)

* do not snapshot input

* do not snapshot input

* Add stubs for rnaseq subworkflows - part IV (#5922)

do not snapshot input with stub

* mulled biocontainer for vardict-java and htslib (#5920)

* vardict-java bgzip mulled container

* update bioconda env

* code polishing

Co-authored-by: Nicolas Vannieuwkerke <[email protected]>

* Update modules/nf-core/vardictjava/main.nf

Co-authored-by: Nicolas Vannieuwkerke <[email protected]>

* args3 for bgzip

* args3

* Update snap

* removed whitespace

---------

Co-authored-by: Nicolas Vannieuwkerke <[email protected]>
Co-authored-by: Simon Pearce <[email protected]>

* Add same file clash check to bioawk (#5926)

* Update pre-commit hook astral-sh/ruff-pre-commit to v0.5.1 (#5893)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Update actions/upload-artifact digest to 0b2256b (#5927)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* dysgu==1.6.4_version update (#5928)

* all_files_update

* Update environment.yml

* update_meta.yml

* input==input_bam

* update_snapshot

* meta.yml,tags.yml update

* Update modules/nf-core/dysgu/meta.yml

Co-authored-by: Famke Bäuerle <[email protected]>

* Update modules/nf-core/dysgu/meta.yml

Co-authored-by: Famke Bäuerle <[email protected]>

* Update modules/nf-core/dysgu/meta.yml

Co-authored-by: Famke Bäuerle <[email protected]>

* Update modules/nf-core/dysgu/meta.yml

Co-authored-by: Famke Bäuerle <[email protected]>

* Update modules/nf-core/dysgu/meta.yml

Co-authored-by: Famke Bäuerle <[email protected]>

* update_author

---------

Co-authored-by: Famke Bäuerle <[email protected]>

* ADD argNorm module (#5905)

* ADD argnorm module

* ENH correct pattern for 'hamronized' input

* ENH specify hamronization command in script

* ENH don't pass 'hamronized' as val

* Fixed trailing whitespace

* "single_end" is not relevant in this module

* alignment

* "db_args" to avoid any confusion with the input channel

* Added a test to check $tool is provided

* Implemented the stub mode

* Added a test for the missing database

* Reordered the arguments to match the script

---------

Co-authored-by: Matthieu Muffato <[email protected]>

* Sentieon license server - testing (#5856)

* test it out

* ci: Add license_message script

nf-core/sarek#1380
https://github.com/DonFreed/docker-actions-test?tab=readme-ov-file

* test: Add tests for sentieon bwamem

Because the pytest-workflow tests are taking so long

* Add scratch from meeting

* Clean up sentieon secrets

* ci: Fix Nextflow secrets setup

* fix(sentieon): Remove encryption key because it won't get used

* fix #

* ci: Fix when to setup secrets

* feat: Add sentieon auth mech secret

* chore(sentieon): Remove bwamem pytest-workflow tests

* fix: Spike in with environment variables

* fix: Add SENTIEON_AUTH_DATA in env file

The reason for this is because when we require it in the module it
fails. That doesn't make sense for normal Sentieon users. They don't
care, this is more of a GitHub actions thing.

* docs(sentieon): Make this a README

* test: Add local testing setup

* test: Remove publishDir

* style: Remove sentieon/bwamem

* fix: Change comments to prints for debugging and better reporting

* fix: Add a quote?

* chore: Remove unnecessary variable

* debug: Print variables

* Fix typo

Co-authored-by: Anders Sune Pedersen <[email protected]>

* chore: Clean up code

* docs: Write up all the learnings from Sentieon

* style: Run prettier

* style: Add tags for nf-core lint

* docs: Clean up notes

Co-authored-by: maxulysse <[email protected]>

* chore: Add @DonFreed as maintainer

---------

Co-authored-by: Edmund Miller <[email protected]>
Co-authored-by: Edmund Miller <[email protected]>
Co-authored-by: Anders Sune Pedersen <[email protected]>
Co-authored-by: maxulysse <[email protected]>

* Fix Sentieon CI (#5934)

* ci: Remove setting secrets in pytest-workflow

* ci: Add a longer note about conditionals

https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions#using-secrets-in-a-workflow
Secrets cannot be directly referenced in if: conditionals.
Instead, consider setting secrets as job-level environment variables, then referencing the environment variables to conditionally run steps in the job.
For more information, see "Contexts" and jobs.<job_id>.steps[*].if.

* ci: Check for secrets in env

Because if this is coming from a fork it will fail

* disable conda setup when not using a conda profile (#5935)

* stubs and test for rnaseq modules - part III (#5925)

* improve bbmap_split stub

* update stub tests

* improve stub

* single snapshot

* fix stub + add stub tests

* fix stub + add stub tests

* fix stub

* restore tags

* fix stub + better stub tests

* fix snapshot

* NARFMAP segfault (#4895)

* test(narfmap): Add segfault test

* fix(narfmap): Clean up log error

* test(narfmap): Should segfault

* chore(narfmap): Bump to 1.4.2

* test(narfmap): Mark test as passing

bioinformaticsorphanage/NARFMAP#10

* style: Add extra tag for nf-core/lint

* test: Update all the snapshot declarations

* chore: Add a TODO for poddar's test

* narfmap cleanup (#5937)

* chore: Add Note about issue

* test: Remove bam name

Co-authored-by: maxulysse <[email protected]>

---------

Co-authored-by: maxulysse <[email protected]>

* Add CRAM/index support to NARFMAP (#5946)

* feat: Add cram/index support to narfmap

* chore: Remove singularity image comment

* test: Update tests for CRAM

* Update actions/setup-node digest to 1e60f62 (#5947)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Fix bcftools plugin split (#5942)

* bump bclconvert (#5952)

* bump bclconvert

* fix snapshots

* bump multiqc (#5950)

* bump multiqc

* update snapshot

* chore: bump picard (#5948)

* bump picard

* fix hsmetrics test

* fix markduplicates

* fix extractfingerprints

* fix addorreplacereadgroups

* fix crosscheckfingerprints

* Add prefix to output file for barrnap (#5919)

* Add prefix to output file

* Update barrnap to nf-test

* Update tags.yml

* Add missing bracket and run test

* Correct format of license

* Update modules/nf-core/barrnap/tests/main.nf.test

Co-authored-by: Simon Pearce <[email protected]>

* Update modules/nf-core/barrnap/tests/main.nf.test

Co-authored-by: Simon Pearce <[email protected]>

* Review suggestions

* Prettier

* Update meta.yml

* Update tests

---------

Co-authored-by: Simon Pearce <[email protected]>

* chore(deps): update actions/setup-python digest to 39cd149 (#5961)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* LAST alignment statistics for MultiQC (#5902)

* last train modified to output multiqc

* Report alignment length and percent similarity, for MultiQC.

* Update snapshot file.

* Document the new channels

* Fix indentation

* Update test results.

* Give a longer suffix for better MultiQC search patterns.

---------

Co-authored-by: Mohammed Mahdi <[email protected]>

* Update lofreq somatic and callparallel  (#5931)

* update lofreq callparallel and somatic ssh

* [automated] Fix linting with Prettier

* suggestions

* remove files tests

* label lofreq somatic

* Update modules/nf-core/lofreq/callparallel/main.nf

Co-authored-by: Famke Bäuerle <[email protected]>

* port lofreq/callparallel to nf-test

* Swap to nftest

* update paths

* add meta

* Update main.nf

* modify main

* update paths

* fix small errors

* Update modules/nf-core/lofreq/somatic/tests/main.nf.test

Co-authored-by: Simon Pearce <[email protected]>

* Update main.nf.test.snap

---------

Co-authored-by: AitorOP <[email protected]>
Co-authored-by: nf-core-bot <[email protected]>
Co-authored-by: Famke Bäuerle <[email protected]>
Co-authored-by: Francisco Martínez <[email protected]>
Co-authored-by: Famke Bäuerle <[email protected]>
Co-authored-by: Simon Pearce <[email protected]>

* Update dupradar.r according to new MultiQC (>=v1.22) config values (#5943)

Update dupradar.r according to new MultiQC config values.

Co-authored-by: Matthias Zepper <[email protected]>

* Update the module: nanoq (#5964)

* update nanoq module

* update nf-test

* Update Glimpse2 phase (#5944)

* Update Glimpse2 phase

* Fix label

* Update snapshot

* Update modules/nf-core/glimpse2/phase/main.nf

Co-authored-by: Maxime U Garcia <[email protected]>

* Update meta.yml

---------

Co-authored-by: LouisLeNezet <[email protected]>
Co-authored-by: Maxime U Garcia <[email protected]>
Co-authored-by: Maxime U Garcia <[email protected]>

* Update dupradar test snapshot (#5966)

* Update dupradar test snapshot

* Dummy commit to try and trigger nf-test in ci

* Revert "Dummy commit to try and trigger nf-test in ci"

This reverts commit 130f998.

* stubs and test for rnaseq modules - part IV (#5939)

* fix stubs and add stub tests

* fix linting

* fix some linting

* fix linting

* better comment

* change container

* update snapshots

* improve untar stubs

* update snapshots

* update snapshots

* update env

* better

* use seqera containers

* quay.io

* proper usage

* update snapshots

* keep ubuntu for now

* update tests and snap

* update test and snapshot

* update test and snapshot

* Add new input to multiqc module for use with --rename_samples (#5973)

* Add new input to multiqc module for use with --rename_samples

* Update tests

* update meta.yml

* Add provision for sample_names

* port `gridss/gridss` to nf-test (#5933)

* port

* remove pytest files

* fix meta

* Update meta.yml

* fix indentation

* update test with bam plugin

* fix indents, try other test

* remove --assembly flag for now

---------

Co-authored-by: Maxime U Garcia <[email protected]>

* by remove the 2>&1 to avoid the warning message. (#5974)

* chore(deps): update pre-commit hook astral-sh/ruff-pre-commit to v0.5.2 (#5975)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Handle empty files (#5720)

* fix: emit all fastqs, remove global log file

* fix: remove rg parseing

* fix(subworkflow/bcl_demultiplex): remove rg parsing, output empty fastqs, remove log file

* test: update snaps

* lint: traill\ing whitespace

* feat: include read group parsing and empty file check in single closure

* test: update snapshot

* fix: lint final new line

* fix: lint trailing whitespace

* fix: add branch and emit empty fastq channel

* tests: update snaps

* lint: trailing whitespace

* fix: filter bool and add test

* test: update snaps

* test: assert empty file exist

* test: assert empty file exist

* test: check file exists, update snap

* Update test.yml

* Update test.yml

* ci: debug nf-test failure

* ci: debug nf-test failure

* ci: debug nf-test failure

* ci: debug nf-test failure

* ci: debug nf-test failure

* ci: debug nf-test failure

* ci: debug nf-test failure

* ci: debug nf-test failure

* ci: debug nf-test failure

* ci: debug nf-test failure

* ci: debug nf-test failure

* ci: debug nf-test failure

* ci: debug nf-test failure

* ci: debug nf-test failure

* ci: debug nf-test failure

* ci: debug nf-test failure

* ci: debug nf-test failure

* ci: debug nf-test failure

* ci: debug nf-test failure

* ci: debug nf-test failure

* ci: debug nf-test failure

* lint: fix lint

---------

Co-authored-by: [email protected] <--unset>
Co-authored-by: Simon Pearce <[email protected]>

* new module: custom_gtffilter (#5965)

* Add basic gtffilter implementation

* Add gtffilter stub

* Add gtffilter meta map

* Update gtffilter meta.yml

* Fix gtffilter formatting

* Add first gtffilter tests

* Fix problems in gtffilter module defintion

* Add gtffilter nf-test snapshot

* Add support for gzipped inputs to gtffilter

* Add gtffilter gzip stub

* Update nf-test definition for gtffilter

* Add meta2 to fasta input

Co-authored-by: Nicolas Vannieuwkerke <[email protected]>

* Add fasta meta map to gtffilter test definitions

* Add gtffilter MIT license text

---------

Co-authored-by: Nicolas Vannieuwkerke <[email protected]>

* Fix gtffilter output definition (#5977)

* Added agat/spmergeannotations (#5772)

* Added agat/spmergeannotations

* Added tests with config

* Rename preprocess rnaseq (#5959)

* Edit content with subworkflow name

* Rename subworkflow folder

* Readd subworkflow with old name, with deprecation

* remove rogue word

* Appease eclint

* Remove test suite for deprecated subworkflow

* Using Seqera containers for Sentieon to avoid LD_LIBRARY errors (#5823)

* Using seqera containers for sentieon

* update containers for sentieon modules with quay.io registry

---------

Co-authored-by: Simon Pearce <[email protected]>
Co-authored-by: Adam Talbot <[email protected]>
Co-authored-by: maxulysse <[email protected]>

* Swap unzip to nf-test (continuation of 5357) (#5897)

* unzip stub, nf-test, snap

* Revert "unzip stub, nf-test, snap"

This reverts commit dec35aa.

* Reapply "unzip stub, nf-test, snap"

This reverts commit e0d708e.

* added minimal required tests for process and stub

* added snapshot

* simplify snapshot input in stub test

Co-authored-by: Matthias Hörtenhuber <[email protected]>

* remove unneeded stub files

* migrate pytest for unzipfiles

* add snapshot file for script block

* added stub

* added stub and stub test

* updated stub snapshot for unzip

* remove unneeded test config

* Update snaps and malt

* Update malt for linting

* Fix linting for maltextract

* Swap test-data paths for malt modules

* Update modules/nf-core/maltextract/tests/main.nf.test

---------

Co-authored-by: Sateesh <[email protected]>
Co-authored-by: Jenny Leopoldina Smith <[email protected]>
Co-authored-by: Matthias Hörtenhuber <[email protected]>
Co-authored-by: Maxime U Garcia <[email protected]>

* Improve strandedness derivation in rnaseq preprocessing swf (#5982)

* Improve strandedness devivation

* Add missing params to nf-test

* RNAseq preprocessing: rrna sequences make more sense as a channel of FASTAs, port tests (#5988)

* rrna sequences make more sense as a channel of FASTAs

* Also update tests

* Add function test to swf

* Fix up subworkflow function testing

* [Upgrade] Normalize input of lofreq-viterbi to match other lofreq modules (#5989)

* add meta2 to input fasta channel

* update snapshot; nothing new but timestamp

* Update modules/nf-core/lofreq/viterbi/tests/main.nf.test

Co-authored-by: Felix Lenner <[email protected]>

* Update modules/nf-core/lofreq/viterbi/tests/main.nf.test

Co-authored-by: Felix Lenner <[email protected]>

* Update modules/nf-core/lofreq/viterbi/tests/main.nf.test

Co-authored-by: Felix Lenner <[email protected]>

---------

Co-authored-by: Felix Lenner <[email protected]>

* Add module fgbio/collectduplexseqmetrics (#5960)

* Initial commit

* Rerun nf-test

* Change assertion for duplex_qc output

* Add ggplot2 to version.yml

* Update snapshot

* Correct conda packages version

* Add Seqfu/derep (#5958)

* add-usearch-unoise3

* add seqfu/derep

* Delete modules/nf-core/usearch/unoise3/environment.yml

* Delete modules/nf-core/usearch/unoise3/main.nf

* remove files

* update metadata

* Update modules/nf-core/seqfu/derep/main.nf

Co-authored-by: Simon Pearce <[email protected]>

* Update modules/nf-core/seqfu/derep/main.nf

Co-authored-by: Simon Pearce <[email protected]>

* Update modules/nf-core/seqfu/derep/main.nf

Co-authored-by: Simon Pearce <[email protected]>

* Update modules/nf-core/seqfu/derep/main.nf

Co-authored-by: Simon Pearce <[email protected]>

* Update modules/nf-core/seqfu/derep/main.nf

Co-authored-by: Simon Pearce <[email protected]>

* Update modules/nf-core/seqfu/derep/tests/main.nf.test

Co-authored-by: Simon Pearce <[email protected]>

* Fix dataset in -stub

* Clarify description

* Update modules/nf-core/seqfu/derep/meta.yml

Co-authored-by: Simon Pearce <[email protected]>

* check for pre-existing output file && update test

* Update main.nf

---------

Co-authored-by: Simon Pearce <[email protected]>

* modify valCutoff funtion in propr.R (#5995)

Co-authored-by: Cristina Araiz <[email protected]>

* liftoff: Re-added conda environment (#5981)

* liftoff: Readded conda environment

* Added liftoff to env file

* Cleaned up env file

* chore(deps): update pre-commit hook astral-sh/ruff-pre-commit to v0.5.3 (#5996)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Update: `CIRCULARMAPPER_CIRCULARGENERATOR` (#5997)

* Fix output naming, add req argument to input

* add new input to nf-test

* update nf-test due to changes in upstream module inputs

* update snapshots

* add metas to other input channels

* update tests according to changes in module inputs.

* Apply suggestions from code review

Co-authored-by: James A. Fellows Yates <[email protected]>

* rename elongation factor for consistency with other CM module

---------

Co-authored-by: James A. Fellows Yates <[email protected]>

* Update: `CIRCULARMAPPER_REALIGNSAMFILE` (#5998)

* Fix output naming, add req argument to input

* add new input to nf-test

* update nf-test due to changes in upstream module inputs

* update snapshots

* add metas to other input channels

* update tests according to changes in module inputs.

* add meta to elongation factor channel for consistency with other CM module

* update nf-test to include new meta

* rename metas

* Add stub and test to lofreq/indelqual (#5941)

* Add stub and test to lofreq/indelqual

* Add author

* new test setup using --migrate-pytest and PR feedback

* Update main.nf.test

* Update meta.yml

* Update main.nf

* add snap; del pytest

* Update meta.yml

* Update main.nf.test

* Update meta.yml

* update test

* minor fix in stub test

* Update tabix and add threads (#5999)

* Update tabix and add threads

* fix linting

* Remove problematic ifEmpty() from rnaseq preprocessing (#6000)

* Refactor Sentieon (#5945)

* refactor: Clean up the logic and let Sentieon figure it out

* refactor: Remove Sentieon license server IP check

* fix: Add a -n

* Try accessing secrets in the module?

* fix: Use Nextflow for if statement

* update snapshot

* update snapshot

* extend logic to all main.nf

* ci: Add Sentieon logic to pytest-workflow

* test(sentieon): Add env section to every config

* Revert "ci: Add Sentieon logic to pytest-workflow"

This reverts commit 76884d2.

* test: Remove pytest_modules for sentieon

---------

Co-authored-by: maxulysse <[email protected]>

* ltrharvest: Added test to verify graceful handling of no ltr input (#5978)

* ltrharvest: Added test to verify graceful handling of no ltr input

* Updated test data paths

* Fixed test data path

* Removed unstable items from snapshot

* Updated snapshotting logic

* ltrfinder: Added test to verify graceful handling of no ltr input (#5979)

* ltrretriever/ltrretriever: Added test to verify graceful handling of … (#5980)

* ltrretriever/ltrretriever: Added test to verify graceful handling of no ltr input

* Removed unstable items from snapshot

* update nf-test t0 0.9.0 (#6010)

* Fix keyword meta (#6011)

* Updated mkfastq module (#5984)

* Updated mkfastq module

* Combined input channels

* Updated main.nf.test and snap

---------

Co-authored-by: zxBIB Schcolnicov <[email protected]>
Co-authored-by: Anabella Trigila <[email protected]>

* bulk update of meta.yml modules structure

* update meta.yml files automatically with manual curation for missing information

---------

Co-authored-by: James A. Fellows Yates <[email protected]>
Co-authored-by: Carson J Miller <[email protected]>
Co-authored-by: Maxime U Garcia <[email protected]>
Co-authored-by: nf-core-bot <[email protected]>
Co-authored-by: Ramprasad Neethiraj <[email protected]>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Priyanka Surana <[email protected]>
Co-authored-by: Matthieu Muffato <[email protected]>
Co-authored-by: Aida Andrades Valtueña <[email protected]>
Co-authored-by: Simon Pearce <[email protected]>
Co-authored-by: Usman Rashid <[email protected]>
Co-authored-by: Anders Sune Pedersen <[email protected]>
Co-authored-by: Maxime U Garcia <[email protected]>
Co-authored-by: Felix Lenner <[email protected]>
Co-authored-by: Christian Heide <[email protected]>
Co-authored-by: Famke Bäuerle <[email protected]>
Co-authored-by: Raquel Manzano <[email protected]>
Co-authored-by: Matthias De Smet <[email protected]>
Co-authored-by: Jonathan Manning <[email protected]>
Co-authored-by: Maxime Borry <[email protected]>
Co-authored-by: vickylaram <[email protected]>
Co-authored-by: Tyler Chafin <[email protected]>
Co-authored-by: Thanh Lee <[email protected]>
Co-authored-by: Joon Klaps <[email protected]>
Co-authored-by: John Palmer <[email protected]>
Co-authored-by: John Palmer <[email protected]>
Co-authored-by: Charles Plessy <[email protected]>
Co-authored-by: Mahesh Binzer-Panchal <[email protected]>
Co-authored-by: Jose Espinosa-Carrasco <[email protected]>
Co-authored-by: thomgiles <[email protected]>
Co-authored-by: Jonathan Manning <[email protected]>
Co-authored-by: Matthieu Muffato <[email protected]>
Co-authored-by: Edmund Miller <[email protected]>
Co-authored-by: Felix Lenner <[email protected]>
Co-authored-by: paulwolk <[email protected]>
Co-authored-by: Lili Andersson-Li <[email protected]>
Co-authored-by: Evangelos Karatzas <[email protected]>
Co-authored-by: Sateesh_Peri <[email protected]>
Co-authored-by: Zohaib Anwar <[email protected]>
Co-authored-by: Gisela Gabernet <[email protected]>
Co-authored-by: Sateesh <[email protected]>
Co-authored-by: Nils Homer <[email protected]>
Co-authored-by: Nicolas Vannieuwkerke <[email protected]>
Co-authored-by: RuthEberhardt <[email protected]>
Co-authored-by: Theresa Störiko <[email protected]>
Co-authored-by: Theresa Störiko <[email protected]>
Co-authored-by: eolaniru <[email protected]>
Co-authored-by: poddarharsh15 <[email protected]>
Co-authored-by: Vedanth Ramji <[email protected]>
Co-authored-by: Edmund Miller <[email protected]>
Co-authored-by: maxulysse <[email protected]>
Co-authored-by: Sofia Stamouli <[email protected]>
Co-authored-by: Mohammed Mahdi <[email protected]>
Co-authored-by: AitorPeseta <[email protected]>
Co-authored-by: AitorOP <[email protected]>
Co-authored-by: Francisco Martínez <[email protected]>
Co-authored-by: Famke Bäuerle <[email protected]>
Co-authored-by: Matthias Zepper <[email protected]>
Co-authored-by: Matthias Zepper <[email protected]>
Co-authored-by: Louis LE NEZET <[email protected]>
Co-authored-by: LouisLeNezet <[email protected]>
Co-authored-by: JIANHONG OU <[email protected]>
Co-authored-by: Kyle Hazen <[email protected]>
Co-authored-by: Nico Trummer <[email protected]>
Co-authored-by: Adam Talbot <[email protected]>
Co-authored-by: Jenny Leopoldina Smith <[email protected]>
Co-authored-by: Matthias Hörtenhuber <[email protected]>
Co-authored-by: Thomas Krannich <[email protected]>
Co-authored-by: Georgia Kesisoglou <[email protected]>
Co-authored-by: Andrea Telatin <[email protected]>
Co-authored-by: caraiz2001 <[email protected]>
Co-authored-by: Cristina Araiz <[email protected]>
Co-authored-by: Thiseas C. Lamnidis <[email protected]>
Co-authored-by: Nicolás Schcolnicov <[email protected]>
Co-authored-by: zxBIB Schcolnicov <[email protected]>
Co-authored-by: Anabella Trigila <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants