Check and expand Ci tests #93

tillenglert · 2023-08-03T13:03:40Z

I added some new tests with already existing data and changed some input sample sheets to have the following test:

without NCBI credentials:

test_full
test_assembly_only
test_bins_only
test_coassembly
test_mouse_allow_all_pep_lengths
test_model_information
test_mhcflurry
test_mhcnuggets_1
test_mhcnuggets_2

with NCBI credentials:

test
test_taxa_only
test_mouse
test_tiny -> I'm not quite sure if we should keep this test, I would personally delete it, as it doesn't give any more information compared to the other tests (@skrakau)

I also opened a PR on test-datasets (nf-core/test-datasets#982) to add the new sample sheets.

PR checklist

…peptide lengths automatically to test this functionality

github-actions · 2023-08-03T13:06:02Z

`nf-core lint` overall result: Failed ❌

Posted for pipeline commit c51f7ad

+| ✅ 151 tests passed       |+
!| ❗   9 tests had warnings |!
-| ❌   5 tests failed       |-

❌ Test failures:

files_unchanged - CODE_OF_CONDUCT.md does not match the template
files_unchanged - .github/CONTRIBUTING.md does not match the template
files_unchanged - .github/workflows/linting.yml does not match the template
files_unchanged - lib/NfcoreTemplate.groovy does not match the template
multiqc_config - 'assets/multiqc_config.yml' does not contain a matching 'report_comment'.

❗ Test warnings:

files_exist - File not found: conf/igenomes.config
readme - README contains the placeholder zenodo.XXXXXXX. This should be replaced with the zenodo doi (after the first release).
pipeline_todos - TODO string in README.md: Add full-sized test dataset and amend the paragraph below if applicable
pipeline_todos - TODO string in README.md: If applicable, make list of people who have also contributed
pipeline_todos - TODO string in README.md: Add citation for pipeline after first release. Uncomment lines below and update Zenodo doi and badge at the top of this file.
pipeline_todos - TODO string in awsfulltest.yml: You can customise AWS full pipeline tests as required
pipeline_todos - TODO string in WorkflowMain.groovy: Add Zenodo DOI for pipeline after first release
pipeline_todos - TODO string in usage.md: Add documentation about anything specific to running your pipeline. For general topics, please point to (and add to) the main nf-core website.
pipeline_todos - TODO string in output.md: Write this documentation describing your workflow's output

✅ Tests passed:

files_exist - File found: .gitattributes
files_exist - File found: .gitignore
files_exist - File found: .nf-core.yml
files_exist - File found: .editorconfig
files_exist - File found: .prettierignore
files_exist - File found: .prettierrc.yml
files_exist - File found: CHANGELOG.md
files_exist - File found: CITATIONS.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_exist - File found: nextflow_schema.json
files_exist - File found: nextflow.config
files_exist - File found: README.md
files_exist - File found: .github/.dockstore.yml
files_exist - File found: .github/CONTRIBUTING.md
files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
files_exist - File found: .github/ISSUE_TEMPLATE/config.yml
files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
files_exist - File found: .github/workflows/branch.yml
files_exist - File found: .github/workflows/ci.yml
files_exist - File found: .github/workflows/linting_comment.yml
files_exist - File found: .github/workflows/linting.yml
files_exist - File found: assets/email_template.html
files_exist - File found: assets/email_template.txt
files_exist - File found: assets/sendmail_template.txt
files_exist - File found: assets/nf-core-metapep_logo_light.png
files_exist - File found: conf/modules.config
files_exist - File found: conf/test.config
files_exist - File found: conf/test_full.config
files_exist - File found: docs/images/nf-core-metapep_logo_light.png
files_exist - File found: docs/images/nf-core-metapep_logo_dark.png
files_exist - File found: docs/output.md
files_exist - File found: docs/README.md
files_exist - File found: docs/README.md
files_exist - File found: docs/usage.md
files_exist - File found: lib/nfcore_external_java_deps.jar
files_exist - File found: lib/NfcoreTemplate.groovy
files_exist - File found: lib/Utils.groovy
files_exist - File found: lib/WorkflowMain.groovy
files_exist - File found: main.nf
files_exist - File found: assets/multiqc_config.yml
files_exist - File found: conf/base.config
files_exist - File found: .github/workflows/awstest.yml
files_exist - File found: .github/workflows/awsfulltest.yml
files_exist - File found: lib/WorkflowMetapep.groovy
files_exist - File found: modules.json
files_exist - File found: pyproject.toml
files_exist - File not found check: Singularity
files_exist - File not found check: parameters.settings.json
files_exist - File not found check: pipeline_template.yml
files_exist - File not found check: .nf-core.yaml
files_exist - File not found check: bin/markdown_to_html.r
files_exist - File not found check: conf/aws.config
files_exist - File not found check: .github/workflows/push_dockerhub.yml
files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
files_exist - File not found check: docs/images/nf-core-metapep_logo.png
files_exist - File not found check: .markdownlint.yml
files_exist - File not found check: .yamllint.yml
files_exist - File not found check: lib/Checks.groovy
files_exist - File not found check: lib/Completion.groovy
files_exist - File not found check: lib/Workflow.groovy
files_exist - File not found check: .travis.yml
nextflow_config - Config variable found: manifest.name
nextflow_config - Config variable found: manifest.nextflowVersion
nextflow_config - Config variable found: manifest.description
nextflow_config - Config variable found: manifest.version
nextflow_config - Config variable found: manifest.homePage
nextflow_config - Config variable found: timeline.enabled
nextflow_config - Config variable found: trace.enabled
nextflow_config - Config variable found: report.enabled
nextflow_config - Config variable found: dag.enabled
nextflow_config - Config variable found: process.cpus
nextflow_config - Config variable found: process.memory
nextflow_config - Config variable found: process.time
nextflow_config - Config variable found: params.outdir
nextflow_config - Config variable found: params.input
nextflow_config - Config variable found: params.validationShowHiddenParams
nextflow_config - Config variable found: params.validationSchemaIgnoreParams
nextflow_config - Config variable found: manifest.mainScript
nextflow_config - Config variable found: timeline.file
nextflow_config - Config variable found: trace.file
nextflow_config - Config variable found: report.file
nextflow_config - Config variable found: dag.file
nextflow_config - Config variable (correctly) not found: params.nf_required_version
nextflow_config - Config variable (correctly) not found: params.container
nextflow_config - Config variable (correctly) not found: params.singleEnd
nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
nextflow_config - Config variable (correctly) not found: params.name
nextflow_config - Config variable (correctly) not found: params.enable_conda
nextflow_config - Config timeline.enabled had correct value: true
nextflow_config - Config report.enabled had correct value: true
nextflow_config - Config trace.enabled had correct value: true
nextflow_config - Config dag.enabled had correct value: true
nextflow_config - Config manifest.name began with nf-core/
nextflow_config - Config variable manifest.homePage began with https://github.com/nf-core/
nextflow_config - Config dag.file ended with .html
nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
nextflow_config - Config manifest.version ends in dev: 1.0dev
nextflow_config - Config params.custom_config_version is set to master
nextflow_config - Config params.custom_config_base is set to https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Lines for loading custom profiles found
files_unchanged - .gitattributes matches the template
files_unchanged - .prettierrc.yml matches the template
files_unchanged - LICENSE matches the template
files_unchanged - .github/.dockstore.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/bug_report.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/config.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
files_unchanged - .github/PULL_REQUEST_TEMPLATE.md matches the template
files_unchanged - .github/workflows/branch.yml matches the template
files_unchanged - .github/workflows/linting_comment.yml matches the template
files_unchanged - assets/email_template.html matches the template
files_unchanged - assets/email_template.txt matches the template
files_unchanged - assets/sendmail_template.txt matches the template
files_unchanged - assets/nf-core-metapep_logo_light.png matches the template
files_unchanged - docs/images/nf-core-metapep_logo_light.png matches the template
files_unchanged - docs/images/nf-core-metapep_logo_dark.png matches the template
files_unchanged - docs/README.md matches the template
files_unchanged - lib/nfcore_external_java_deps.jar matches the template
files_unchanged - .gitignore matches the template
files_unchanged - .prettierignore matches the template
files_unchanged - pyproject.toml matches the template
actions_ci - '.github/workflows/ci.yml' is triggered on expected events
actions_ci - '.github/workflows/ci.yml' checks minimum NF version
actions_awstest - '.github/workflows/awstest.yml' is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml does not use -profile test
readme - README Nextflow minimum version badge matched config. Badge: 23.04.0, Config: 23.04.0
pipeline_name_conventions - Name adheres to nf-core convention
template_strings - Did not find any Jinja template strings (134 files)
schema_lint - Schema lint passed
schema_lint - Schema title + description lint passed
schema_lint - Input mimetype lint passed: 'text/csv'
schema_params - Schema matched params returned from nextflow config
system_exit - No System.exit calls found
actions_schema_validation - Workflow validation passed: ci.yml
actions_schema_validation - Workflow validation passed: fix-linting.yml
actions_schema_validation - Workflow validation passed: awsfulltest.yml
actions_schema_validation - Workflow validation passed: branch.yml
actions_schema_validation - Workflow validation passed: linting.yml
actions_schema_validation - Workflow validation passed: clean-up.yml
actions_schema_validation - Workflow validation passed: awstest.yml
actions_schema_validation - Workflow validation passed: linting_comment.yml
merge_markers - No merge markers found in pipeline files
modules_json - Only installed modules found in modules.json
multiqc_config - 'assets/multiqc_config.yml' follows the ordering scheme of the minimally required plugins.
multiqc_config - 'assets/multiqc_config.yml' contains 'export_plots: true'.
modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'

Run details

nf-core/tools version 2.10
Run at 2023-10-02 07:23:00

…onally added assembly only

tillenglert · 2023-08-04T05:21:20Z

@nf-core-bot fix linting

conf/test_mhcflurry.config

conf/test_mhcnuggets_1.config

conf/test_mhcnuggets_2.config

skrakau · 2023-09-11T12:26:38Z

.github/workflows/ci.yml

@@ -71,7 +80,7 @@ jobs:
    strategy:
      fail-fast: false
      matrix:
-        tests: ["test", "test_mouse", "test_tiny"] # add further test profiles here, will be run in parallel (but only with one nextflow version)
+        tests: ["test", "test_mouse", "test_tiny", "test_taxa_only"] # add further test profiles here, will be run in parallel (but only with one nextflow version)


Could you change the order here (it was changed in some refactoring in the past), such that the profile test is used in the test above and the others here as additional? would seem more intuitive, or is there still a reason why it is like it is?

doesn't work due to credentials ...

Co-authored-by: Sabrina Krakau <[email protected]>

tillenglert · 2023-09-14T06:52:31Z

I added the versioning directory on test-datasets, so all tests will fail until this is merged.

tillenglert · 2023-09-14T07:26:03Z

@nf-core-bot fix linting

tillenglert · 2023-09-27T05:30:06Z

@skrakau linting is still failing due to template update, but it still expects test_bins! I will change it when this PR is merged or would you first change the expected tests for branch protection before merging?

tillenglert · 2023-10-02T07:42:02Z

@skrakau Maybe different question: Which tests should I make required? All of the new/old ones? :)

skrakau · 2023-10-11T07:18:57Z

would there be a reason for any of the tests to not make them required? Otherwise I would make them required

skrakau · 2023-10-11T07:19:26Z

@skrakau linting is still failing due to template update, but it still expects test_bins! I will change it when this PR is merged or would you first change the expected tests for branch protection before merging?

it's fine to do it afterwards, if you can? Maybe would be easier before

tillenglert · 2023-10-11T07:20:05Z

would there be a reason for any of the tests to not make them required? Otherwise I would make them required

No not really, okay I will make them required! They should all pass if one makes a PR! :)

skrakau

LGTM!

tillenglert added 10 commits August 2, 2023 15:18

Add test to allow all peptide lengths for mouse test

488ce19

Change larger mosue test including also taxa to reduce the number of …

e02e13a

…peptide lengths automatically to test this functionality

Add test taxa only

3ae3c66

add test bins only

a88ebc0

Add tests to gh ci

432c4d3

Add correct name and remove ncbi from test all peptide lengths

c6ba6ed

Add test taxa to tests run with gh secrets

e859215

Add comment to add ncbi mail and pw

119a158

remove todos in test profile

e1c5c07

Add tests for mhcflurry and mhcnuggets

39fa300

tillenglert mentioned this pull request Aug 3, 2023

Ci tests cleanup nf-core/test-datasets#982

Merged

tillenglert added 2 commits August 3, 2023 15:09

Adjust line in usage.md for test not needing any configuration additi…

5d65ada

…onally added assembly only

Use mhc2 alleles test for mhcnuggets class 2

b277395

nf-core-bot and others added 4 commits August 4, 2023 05:21

[automated] Fix linting with Prettier

7e65b30

update changelog

9c9cd5f

Update parameter allow_inconsistent_pep_lengths

2968449

Merge branch 'dev' into CI_tests

992820b

tillenglert marked this pull request as ready for review September 11, 2023 09:25

skrakau reviewed Sep 11, 2023

View reviewed changes

conf/test_mhcflurry.config Outdated Show resolved Hide resolved

skrakau reviewed Sep 11, 2023

View reviewed changes

conf/test_mhcnuggets_1.config Outdated Show resolved Hide resolved

skrakau reviewed Sep 11, 2023

View reviewed changes

conf/test_mhcnuggets_2.config Outdated Show resolved Hide resolved

skrakau reviewed Sep 11, 2023

View reviewed changes

tillenglert and others added 4 commits September 11, 2023 14:54

Apply suggestions from code review

8ab5217

Co-authored-by: Sabrina Krakau <[email protected]>

Update conf/test_mhcnuggets_2.config

1dfc608

Co-authored-by: Sabrina Krakau <[email protected]>

Add/change tests according to suggestions

2d34c3e

create versioning directory in test-datasets

8160cfb

Merge branch 'dev' into CI_tests

1fc49b3

nf-core-bot and others added 3 commits September 14, 2023 07:26

[automated] Fix linting with Prettier

151d7e4

Change to samplesheet instead of input

15ec56f

rename samplesheet for mouse assembly only

4d7bd0c

Merge branch 'dev' into CI_tests

c51f7ad

skrakau approved these changes Oct 11, 2023

View reviewed changes

tillenglert merged commit 2e8e11b into nf-core:dev Oct 16, 2023

tillenglert mentioned this pull request Oct 16, 2023

Clean up CI tests #41

Closed

tillenglert deleted the CI_tests branch October 16, 2023 12:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Check and expand Ci tests #93

Check and expand Ci tests #93

tillenglert commented Aug 3, 2023 •

edited

Loading

github-actions bot commented Aug 3, 2023 •

edited

Loading

❌ Test failures:

❗ Test warnings:

✅ Tests passed:

Run details

tillenglert commented Aug 4, 2023

skrakau Sep 11, 2023

skrakau Sep 11, 2023

tillenglert commented Sep 14, 2023

tillenglert commented Sep 14, 2023

tillenglert commented Sep 27, 2023

tillenglert commented Oct 2, 2023

skrakau commented Oct 11, 2023

skrakau commented Oct 11, 2023 •

edited

Loading

tillenglert commented Oct 11, 2023

skrakau left a comment

Check and expand Ci tests #93

Check and expand Ci tests #93

Conversation

tillenglert commented Aug 3, 2023 • edited Loading

PR checklist

github-actions bot commented Aug 3, 2023 • edited Loading

nf-core lint overall result: Failed ❌

❌ Test failures:

❗ Test warnings:

✅ Tests passed:

Run details

tillenglert commented Aug 4, 2023

skrakau Sep 11, 2023

Choose a reason for hiding this comment

skrakau Sep 11, 2023

Choose a reason for hiding this comment

tillenglert commented Sep 14, 2023

tillenglert commented Sep 14, 2023

tillenglert commented Sep 27, 2023

tillenglert commented Oct 2, 2023

skrakau commented Oct 11, 2023

skrakau commented Oct 11, 2023 • edited Loading

tillenglert commented Oct 11, 2023

skrakau left a comment

Choose a reason for hiding this comment

tillenglert commented Aug 3, 2023 •

edited

Loading

github-actions bot commented Aug 3, 2023 •

edited

Loading

`nf-core lint` overall result: Failed ❌

skrakau commented Oct 11, 2023 •

edited

Loading