Dev -> Master for release #289

ewels · 2022-12-15T23:14:06Z

Prepping a new release.

Bump v2.3.0dev

Fix nf-test

Fixes bug where a local file would not work

See https://help.tower.nf/22.3/reports/overview/?h=reports#path-pattern

Wrap `params.fasta` with `file()` when used in processes.

Shouldn't code when tired..

Test samplesheet has been updated to include a mix of SE and PE data.

Also, invalidate test data cache key

First attempt at fetching test data before running nf-test

Samplesheet extra logging

Add a `tower.yml` file to enable Reports in Nextflow Tower

Tools 2.7.1 merge

github-actions · 2022-12-15T23:15:51Z

`nf-core lint` overall result: Passed ✅ ⚠️

Posted for pipeline commit d542bd1

+| ✅ 156 tests passed       |+
!| ❗   1 tests had warnings |!

❗ Test warnings:

nextflow_config - Config manifest.version should end in dev: '2.3.0'

✅ Tests passed:

files_exist - File found: .gitattributes
files_exist - File found: .gitignore
files_exist - File found: .nf-core.yml
files_exist - File found: .editorconfig
files_exist - File found: .prettierignore
files_exist - File found: .prettierrc.yml
files_exist - File found: CHANGELOG.md
files_exist - File found: CITATIONS.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_exist - File found: nextflow_schema.json
files_exist - File found: nextflow.config
files_exist - File found: README.md
files_exist - File found: .github/.dockstore.yml
files_exist - File found: .github/CONTRIBUTING.md
files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
files_exist - File found: .github/ISSUE_TEMPLATE/config.yml
files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
files_exist - File found: .github/workflows/branch.yml
files_exist - File found: .github/workflows/ci.yml
files_exist - File found: .github/workflows/linting_comment.yml
files_exist - File found: .github/workflows/linting.yml
files_exist - File found: assets/email_template.html
files_exist - File found: assets/email_template.txt
files_exist - File found: assets/sendmail_template.txt
files_exist - File found: assets/nf-core-methylseq_logo_light.png
files_exist - File found: conf/modules.config
files_exist - File found: conf/test.config
files_exist - File found: conf/test_full.config
files_exist - File found: docs/images/nf-core-methylseq_logo_light.png
files_exist - File found: docs/images/nf-core-methylseq_logo_dark.png
files_exist - File found: docs/output.md
files_exist - File found: docs/README.md
files_exist - File found: docs/README.md
files_exist - File found: docs/usage.md
files_exist - File found: lib/nfcore_external_java_deps.jar
files_exist - File found: lib/NfcoreSchema.groovy
files_exist - File found: lib/NfcoreTemplate.groovy
files_exist - File found: lib/Utils.groovy
files_exist - File found: lib/WorkflowMain.groovy
files_exist - File found: main.nf
files_exist - File found: assets/multiqc_config.yml
files_exist - File found: conf/base.config
files_exist - File found: conf/igenomes.config
files_exist - File found: .github/workflows/awstest.yml
files_exist - File found: .github/workflows/awsfulltest.yml
files_exist - File found: lib/WorkflowMethylseq.groovy
files_exist - File found: modules.json
files_exist - File found: pyproject.toml
files_exist - File not found check: Singularity
files_exist - File not found check: parameters.settings.json
files_exist - File not found check: .nf-core.yaml
files_exist - File not found check: bin/markdown_to_html.r
files_exist - File not found check: conf/aws.config
files_exist - File not found check: .github/workflows/push_dockerhub.yml
files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
files_exist - File not found check: docs/images/nf-core-methylseq_logo.png
files_exist - File not found check: .markdownlint.yml
files_exist - File not found check: .yamllint.yml
files_exist - File not found check: lib/Checks.groovy
files_exist - File not found check: lib/Completion.groovy
files_exist - File not found check: lib/Workflow.groovy
files_exist - File not found check: .travis.yml
nextflow_config - Config variable found: manifest.name
nextflow_config - Config variable found: manifest.nextflowVersion
nextflow_config - Config variable found: manifest.description
nextflow_config - Config variable found: manifest.version
nextflow_config - Config variable found: manifest.homePage
nextflow_config - Config variable found: timeline.enabled
nextflow_config - Config variable found: trace.enabled
nextflow_config - Config variable found: report.enabled
nextflow_config - Config variable found: dag.enabled
nextflow_config - Config variable found: process.cpus
nextflow_config - Config variable found: process.memory
nextflow_config - Config variable found: process.time
nextflow_config - Config variable found: params.outdir
nextflow_config - Config variable found: params.input
nextflow_config - Config variable found: params.show_hidden_params
nextflow_config - Config variable found: params.schema_ignore_params
nextflow_config - Config variable found: manifest.mainScript
nextflow_config - Config variable found: timeline.file
nextflow_config - Config variable found: trace.file
nextflow_config - Config variable found: report.file
nextflow_config - Config variable found: dag.file
nextflow_config - Config variable (correctly) not found: params.nf_required_version
nextflow_config - Config variable (correctly) not found: params.container
nextflow_config - Config variable (correctly) not found: params.singleEnd
nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
nextflow_config - Config variable (correctly) not found: params.name
nextflow_config - Config timeline.enabled had correct value: true
nextflow_config - Config report.enabled had correct value: true
nextflow_config - Config trace.enabled had correct value: true
nextflow_config - Config dag.enabled had correct value: true
nextflow_config - Config manifest.name began with nf-core/
nextflow_config - Config variable manifest.homePage began with https://github.com/nf-core/
nextflow_config - Config dag.file ended with .html
nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
nextflow_config - Config params.custom_config_version is set to master
nextflow_config - Config params.custom_config_base is set to https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Lines for loading custom profiles found
files_unchanged - .gitattributes matches the template
files_unchanged - .prettierrc.yml matches the template
files_unchanged - CODE_OF_CONDUCT.md matches the template
files_unchanged - LICENSE matches the template
files_unchanged - .github/.dockstore.yml matches the template
files_unchanged - .github/CONTRIBUTING.md matches the template
files_unchanged - .github/ISSUE_TEMPLATE/bug_report.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/config.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
files_unchanged - .github/PULL_REQUEST_TEMPLATE.md matches the template
files_unchanged - .github/workflows/branch.yml matches the template
files_unchanged - .github/workflows/linting_comment.yml matches the template
files_unchanged - .github/workflows/linting.yml matches the template
files_unchanged - assets/email_template.html matches the template
files_unchanged - assets/email_template.txt matches the template
files_unchanged - assets/sendmail_template.txt matches the template
files_unchanged - assets/nf-core-methylseq_logo_light.png matches the template
files_unchanged - docs/images/nf-core-methylseq_logo_light.png matches the template
files_unchanged - docs/images/nf-core-methylseq_logo_dark.png matches the template
files_unchanged - docs/README.md matches the template
files_unchanged - lib/nfcore_external_java_deps.jar matches the template
files_unchanged - lib/NfcoreSchema.groovy matches the template
files_unchanged - lib/NfcoreTemplate.groovy matches the template
files_unchanged - .gitignore matches the template
files_unchanged - .prettierignore matches the template
files_unchanged - pyproject.toml matches the template
actions_ci - '.github/workflows/ci.yml' is triggered on expected events
actions_ci - '.github/workflows/ci.yml' checks minimum NF version
actions_awstest - '.github/workflows/awstest.yml' is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml does not use -profile test
readme - README Nextflow minimum version badge matched config. Badge: 22.10.1, Config: 22.10.1
readme - README Nextflow minimum version in Quick Start section matched config. README: 22.10.1, Config: 22.10.1
pipeline_todos - No TODO strings found
pipeline_name_conventions - Name adheres to nf-core convention
template_strings - Did not find any Jinja template strings (132 files)
schema_lint - Schema lint passed
schema_lint - Schema title + description lint passed
schema_lint - Input mimetype lint passed: 'text/csv'
schema_params - Schema matched params returned from nextflow config
actions_schema_validation - Workflow validation passed: fix-linting.yml
actions_schema_validation - Workflow validation passed: linting_comment.yml
actions_schema_validation - Workflow validation passed: awstest.yml
actions_schema_validation - Workflow validation passed: ci.yml
actions_schema_validation - Workflow validation passed: branch.yml
actions_schema_validation - Workflow validation passed: awsfulltest.yml
actions_schema_validation - Workflow validation passed: linting.yml
merge_markers - No merge markers found in pipeline files
modules_json - Only installed modules found in modules.json
multiqc_config - 'assets/multiqc_config.yml' follows the ordering scheme of the minimally required plugins.
multiqc_config - 'assets/multiqc_config.yml' contains a matching 'report_comment'.
multiqc_config - 'assets/multiqc_config.yml' contains 'export_plots: true'.
modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'

Run details

nf-core/tools version 2.7.1
Run at 2022-12-16 23:13:41

Bump to v2.3.0 in changelog and config

subworkflows/local/bismark.nf

subworkflows/local/prepare_genome.nf

See https://github.com/nf-core/tools/pull/1966/files\#r1051211706

ewels · 2022-12-16T23:11:43Z

Reminder to myself: double check that the new times config is resolving properly.

Checked on Gitpod by faking process.executor = slurm one process config block at a time. Seems to be working fine across the board 👍🏻 I think we're good to go..

Remove .ifEmpty(null)

ewels added 30 commits November 29, 2022 00:25

Bump to v2.3.0dev

51c9ce4

Minor tidying on v2.2.0 changelog

b99b75d

Checkout action v3 everywhere

3d5ea8b

Merge pull request #277 from ewels/bump-v2.3.0dev

4ea9b58

Bump v2.3.0dev

Fix typo in index tests for bwameth

9af8e45

Revert updating checkout action to fix nf-core linting

d014e6e

Check if supplied reference index files exist

7b981cf

Fix some indenation in the config

1894d7d

Get nf-test to use indexes built by first test

d82dc8f

Fix typo in bismark_hisat test path

2268835

Update expected task count for bismark_hisat tests

52fe4dc

Update bwameth test count

8a3eb4e

Mixed up the test numbers for bwameth

648b7ac

Changelog update

4444abc

Merge pull request #279 from nf-core/fix-tests-param-typo

e36e249

Fix nf-test

Wrap params.fasta with file() when used in processes.

5cc2ddd

Fixes bug where a local file would not work

Add a tower.yml file to enable Reports in Nextflow Tower

38a86bc

Update tower.yml

092cb16

Simplify patterns based on docs

4ca7f76

See https://help.tower.nf/22.3/reports/overview/?h=reports#path-pattern

Make report titles shorter

84d7cbe

Reports: split up pre- and post- trimming FastQC reports

86620db

Merge pull request #280 from nf-core/fix-fasta-bug

dadb60d

Wrap `params.fasta` with `file()` when used in processes.

Don't import different subworkflows under same name

419b953

Move bismark index prep + params out of bismark subworkflow

ab082ed

Move genome prep + params out of bwameth subworkflow

ecfe95b

file() can't take 'null' as an input

bf8b74e

Fix module import paths

224240c

Revert earlier change of moving channels to the top.

916cb76

Shouldn't code when tired..

Update after testing by @FelixKrueger

82f4b6b

Move genome index preparation into a subworkflow

bbcf365

ewels and others added 19 commits December 6, 2022 23:44

Give nf-test a config file, even if it's empty

3bf512f

I should go to bed

d9f2228

Remove separate PE test profile.

c36b197

Test samplesheet has been updated to include a mix of SE and PE data.

--profile overwrites nf-test config profile

259c2cc

More logging if samplesheet format autodetection fails

313e464

Don't run CI on dsl2 branch

91c5625

Also, invalidate test data cache key

Remove --debug from nf-test

f4683f5

nf-test: Use snapshots to test number of expected tasks

63002f2

Add params.test_data_base to nextflow schema

6b0f53c

Template update for nf-core/tools version 2.7.1

9d8bc9b

Merge pull request #286 from ewels/actions-pre-pull-test-data

1485a55

First attempt at fetching test data before running nf-test

Merge branch 'dev' into tower-config

abbde77

Merge pull request #284 from ewels/samplesheet-extra-logging

ccd5ac6

Samplesheet extra logging

Merge pull request #281 from ewels/tower-config

fce5bf8

Add a `tower.yml` file to enable Reports in Nextflow Tower

Merge branch 'master' into nf-core-template-merge-2.7.1

7c3ae11

Remove duplicate 'version' from schema

ad6a20a

Update syntax in modules.json

8735b59

Revert unique() syntax change from template

1a1d68f

Merge pull request #288 from ewels/tools-2.7.1-merge

dec70a2

Tools 2.7.1 merge

ewels added 2 commits December 16, 2022 00:17

Bump to v2.3.0 in changelog and config

ee7b4b3

Merge pull request #290 from ewels/v2.3.0-version-bump

8dc9fb3

Bump to v2.3.0 in changelog and config

njspix reviewed Dec 16, 2022

View reviewed changes

subworkflows/local/bismark.nf Show resolved Hide resolved

njspix reviewed Dec 16, 2022

View reviewed changes

subworkflows/local/prepare_genome.nf Show resolved Hide resolved

This comment was marked as resolved.

Sign in to view

njspix approved these changes Dec 16, 2022

View reviewed changes

Remove .ifEmpty(null)

60f5f7a

See https://github.com/nf-core/tools/pull/1966/files\#r1051211706

Merge pull request #291 from ewels/remove-ifempty-null

d542bd1

Remove .ifEmpty(null)

ewels merged commit 93bc581 into master Dec 17, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dev -> Master for release #289

Dev -> Master for release #289

ewels commented Dec 15, 2022

github-actions bot commented Dec 15, 2022 •

edited

Loading

❗ Test warnings:

✅ Tests passed:

Run details

This comment was marked as resolved.

ewels commented Dec 16, 2022

Dev -> Master for release #289

Dev -> Master for release #289

Conversation

ewels commented Dec 15, 2022

github-actions bot commented Dec 15, 2022 • edited Loading

nf-core lint overall result: Passed ✅ ⚠️

❗ Test warnings:

✅ Tests passed:

Run details

This comment was marked as resolved.

ewels commented Dec 16, 2022

github-actions bot commented Dec 15, 2022 •

edited

Loading

`nf-core lint` overall result: Passed ✅ ⚠️