-
Notifications
You must be signed in to change notification settings - Fork 732
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Restructure nf-core/modules repo #2141
Conversation
Has anyone ran |
Doesn't look like Feel free to revert b68b3b7 if that was intentional. |
Good spot. Why was the |
Ah, because that file is only used by the Actions CI? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
💥 🚀
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just left my approval here, not even able to load all the changes 😛
* Move all modules to modules/nf-core/ folder * Move all tests to modules/nf-core/ folder * Replace relative paths in modules test main scripts * Replace relative paths in subworkflow main scripts * Replace relative paths in subworkflow test scripts * Replace paths to tests in test.yml * Standardise path to tests/config to ./tests/config everywhere * Update relative paths in tests * Update paths in pytest_modules.yml Co-authored-by: Edmund Miller <[email protected]>
nf-core#2141 remove files for restructed UniverSC module (avoids duplicate tests)
* initialise template for new module: "universc" * update UniverSC module updates metadata, docker container source, licensing, and citations * update UniverSC module unit tests and documentation * update UniverSC module add inputs, outputs and example calls for UniverSC and Cell Ranger v3.0.2 calls versions for UniverSC and Cell Ranger * initialise template for new module: "universc" * update UniverSC module updates metadata, docker container source, licensing, and citations * update UniverSC module unit tests and documentation * update UniverSC module add inputs, outputs and example calls for UniverSC and Cell Ranger v3.0.2 calls versions for UniverSC and Cell Ranger * resolve formatting issues for UniverSC module * resolve linting errors for UniverSC module * fix test jobs to call UniverSC version without errors * correct configuration for UniverSC test jobs * correct linting errors for UniverSC module * correct docker build files for UniverSC * correct syntax errors in cellranger version call * prettier docs for UniverSC * add output to test data for UniverSC module * update UniverSC module to restructured repo #2141 remove files for restructed UniverSC module (avoids duplicate tests) * define separate outputs for Cell Ranger and UniverSC tests * remove TODO statements and update UniverSC meta.yml defines input and output variables and triggers automated tests * resolve minor linting issues with UniverSC * update paths to UniverSC module in test config * update container versions and tags for UniverSC module * simplifiy container configurations for UniverSC * update configuration for UniverSC tests (build Cell Ranger transcriptome reference first) * test UniverSC module with Cell Ranger references * update tests for UniverSC for restructured repository * update reference inputs for UniverSC module * set up references for cellranger OS test * resolve permissions errors for starting UniverSC * update input arguments for UniverSC and Cell Ranger OS (tests passing) * correct versions and checksums for UniverSC tests * resolves linting issues for UniverSC * resolves linting issues for UniverSC * update expected test outputs for UniverSC tests * migrate UniverSC tests to calling open source Cell Ranger uses Cell Ranger 3.0.2 OS implementation (MIT License) tests passing locally * migrate changes to source code to updated UniverSC container * update unit tests for UniverSC to correct output (using new container) * update output criteria for UniverSC unit tests * change output directory for UniverSC unit tests * test adding podman to GitHub actions (will revert if reviewers object to it) * correct test errors for Cell Ranger OS tests (UniverSC module) * array format for test checks (UniverSC and Cell Ranger OS) * remove unncessary files from UniverSC module * remove podman from automated testing * remove mentions of nf-core/universc container * call executable script from PATH in UniverSC container * migrate UniverSC module Cell Ranger OS count own directory * reorganise UniverSC submodules * update process names in UniverSC module for consistency * update formatting for UniverSC tests * update unit tests for UniverSC submodules running Cell Ranger OS 3.0.2 * reorganise UniverSC submodules to fit naming conventions * remove stub from UniverSC for testing * add universc/mkfastq to unit tests * correct syntax in cellranger module files * update expected output for universc and universc/count tests now consistent with cellranger/count module * correct syntax for universc/count meta.yaml (pass linting) * add stub to universc and universc/count * update unit tests for universc module * update unit tests for universc module * update unit tests for universc module * update unit tests for universc module * update universc/mkfastq test to run stub * update universc/mkfastq expected outputs when running stub * remove trailing whitespace (linting error) * restructure UniverSC main module * update configuration to run each UniverSC test once only * correct UniverSC unit test configuration * update name of tools in universc tests configuration * update expected output for Cell Ranger and UniverSC tests * updates unit tests for Cell Ranger and UniverSC uses contain for web summary HTML as suggested by @apeltzer #1706 * updates unit tests for Cell Ranger for web summary HTML * updates unit tests for Cell Ranger for web summary HTML * updates unit tests for Cell Ranger for web summary HTML to use description * correct path to Cell Ranger test output * update container options for run UniverSC with singularity runs without root priviledges in writeable container * migrate UniverSC container to mirrored image at nfcore/universc:1.2.4 adds documentation for image build configuration discussed in #1706 * remove redundant submodules from UniverSC with functions already supported by Cell Ranger module #1706 * migrate UniverSC references to generate by Cell Ranger submodules * update test configuration for universc/launch * update expected outputs for UniverSC to use Cell Ranger references * update expected outputs for UniverSC to use Cell Ranger references * remove UniverSC submodules for mkref and mkgtf (already implement in Cell Ranger module) discussed in #1706 * move universc/launch submodule to universc module * remove tests for UniverSC submodules for mkref and mkgtf (already implement in Cell Ranger module) discussed in #1706 * move tests for universc/launch submodule to universc module * migrate universc/launch submodule to universc module * update paths in unit tests from universc/launch to universc * update documentation for UniverSC module * update paths in test config from universc/launch to universc * restore cellranger module (remove changes from PR 1706) * restore cellranger module (remove changes from PR 1706) * restore cellranger module (remove changes from PR 1706) * update style of documentation to pass linting * add podman to settings and docs (passes local test) * test podman configuration * test podman configuration * restore changes to testing (removes podman discussed in #2675) * restore changes to other modules (removes cellranger discussed in #2646) * update podman settings in UniverSC docs * update podman parameters * update container version for universc to stable release 1.2.5 * remove conda tests for universc (not supported) * update container version for universc to latest release 1.2.5.1 (run tests on pushed version on personal account) * update container version for universc to use nfcore/universc:1.2.5.1 mirror * exit logic for universc module that doesn't support conda consistent with other modules exit logic for modules that dont support conda #2657 * trigger GitHub Actions test for tomkellygenetics/universc:1.2.5.1 * add log files to universc output directory (confirm running subroutines as expected) * correct UniverSC test configuration addresses singularity test issue https://github.com/nf-core/modules/actions/runs/3955706571/jobs/6774566021 * update configuration for singularity in universc tests * test running universc with singularity --fakeroot requires shadow-uidmap::newuidmap installed * update configuration for singularity in universc tests * debug GH Actions configuration for singularity in universc tests * test running singularity with —fakeroot write permissions * test singularity— * revert changes to singularity tests disables singularity for universc (image too large) * update container settings for universc allows running rootless podman or singularity using --runtime crun or --writable-tmpfs apptainer/singularity#3220 * test universc with singularity --writable-tmpfs * revert changes to singularity tests (--writable-tmpfs not supported on GH Actions) * update container settings for universc to call nfcore/universc:1.2.5.1 (pending mirrored version available) * update version in UniverSC citation --------- Co-authored-by: Simon Thomas Kelly <[email protected]> Co-authored-by: Gisela Gabernet <[email protected]> Co-authored-by: TomKellyGenetics <tomkellygenetics@gmail> Co-authored-by: Alexander Peltzer <[email protected]>
* initialise template for new module: "universc" * update UniverSC module updates metadata, docker container source, licensing, and citations * update UniverSC module unit tests and documentation * update UniverSC module add inputs, outputs and example calls for UniverSC and Cell Ranger v3.0.2 calls versions for UniverSC and Cell Ranger * initialise template for new module: "universc" * update UniverSC module updates metadata, docker container source, licensing, and citations * update UniverSC module unit tests and documentation * update UniverSC module add inputs, outputs and example calls for UniverSC and Cell Ranger v3.0.2 calls versions for UniverSC and Cell Ranger * resolve formatting issues for UniverSC module * resolve linting errors for UniverSC module * fix test jobs to call UniverSC version without errors * correct configuration for UniverSC test jobs * correct linting errors for UniverSC module * correct docker build files for UniverSC * correct syntax errors in cellranger version call * prettier docs for UniverSC * add output to test data for UniverSC module * update UniverSC module to restructured repo nf-core#2141 remove files for restructed UniverSC module (avoids duplicate tests) * define separate outputs for Cell Ranger and UniverSC tests * remove TODO statements and update UniverSC meta.yml defines input and output variables and triggers automated tests * resolve minor linting issues with UniverSC * update paths to UniverSC module in test config * update container versions and tags for UniverSC module * simplifiy container configurations for UniverSC * update configuration for UniverSC tests (build Cell Ranger transcriptome reference first) * test UniverSC module with Cell Ranger references * update tests for UniverSC for restructured repository * update reference inputs for UniverSC module * set up references for cellranger OS test * resolve permissions errors for starting UniverSC * update input arguments for UniverSC and Cell Ranger OS (tests passing) * correct versions and checksums for UniverSC tests * resolves linting issues for UniverSC * resolves linting issues for UniverSC * update expected test outputs for UniverSC tests * migrate UniverSC tests to calling open source Cell Ranger uses Cell Ranger 3.0.2 OS implementation (MIT License) tests passing locally * migrate changes to source code to updated UniverSC container * update unit tests for UniverSC to correct output (using new container) * update output criteria for UniverSC unit tests * change output directory for UniverSC unit tests * test adding podman to GitHub actions (will revert if reviewers object to it) * correct test errors for Cell Ranger OS tests (UniverSC module) * array format for test checks (UniverSC and Cell Ranger OS) * remove unncessary files from UniverSC module * remove podman from automated testing * remove mentions of nf-core/universc container * call executable script from PATH in UniverSC container * migrate UniverSC module Cell Ranger OS count own directory * reorganise UniverSC submodules * update process names in UniverSC module for consistency * update formatting for UniverSC tests * update unit tests for UniverSC submodules running Cell Ranger OS 3.0.2 * reorganise UniverSC submodules to fit naming conventions * remove stub from UniverSC for testing * add universc/mkfastq to unit tests * correct syntax in cellranger module files * update expected output for universc and universc/count tests now consistent with cellranger/count module * correct syntax for universc/count meta.yaml (pass linting) * add stub to universc and universc/count * update unit tests for universc module * update unit tests for universc module * update unit tests for universc module * update unit tests for universc module * update universc/mkfastq test to run stub * update universc/mkfastq expected outputs when running stub * remove trailing whitespace (linting error) * restructure UniverSC main module * update configuration to run each UniverSC test once only * correct UniverSC unit test configuration * update name of tools in universc tests configuration * update expected output for Cell Ranger and UniverSC tests * updates unit tests for Cell Ranger and UniverSC uses contain for web summary HTML as suggested by @apeltzer nf-core#1706 * updates unit tests for Cell Ranger for web summary HTML * updates unit tests for Cell Ranger for web summary HTML * updates unit tests for Cell Ranger for web summary HTML to use description * correct path to Cell Ranger test output * update container options for run UniverSC with singularity runs without root priviledges in writeable container * migrate UniverSC container to mirrored image at nfcore/universc:1.2.4 adds documentation for image build configuration discussed in nf-core#1706 * remove redundant submodules from UniverSC with functions already supported by Cell Ranger module nf-core#1706 * migrate UniverSC references to generate by Cell Ranger submodules * update test configuration for universc/launch * update expected outputs for UniverSC to use Cell Ranger references * update expected outputs for UniverSC to use Cell Ranger references * remove UniverSC submodules for mkref and mkgtf (already implement in Cell Ranger module) discussed in nf-core#1706 * move universc/launch submodule to universc module * remove tests for UniverSC submodules for mkref and mkgtf (already implement in Cell Ranger module) discussed in nf-core#1706 * move tests for universc/launch submodule to universc module * migrate universc/launch submodule to universc module * update paths in unit tests from universc/launch to universc * update documentation for UniverSC module * update paths in test config from universc/launch to universc * restore cellranger module (remove changes from PR 1706) * restore cellranger module (remove changes from PR 1706) * restore cellranger module (remove changes from PR 1706) * update style of documentation to pass linting * add podman to settings and docs (passes local test) * test podman configuration * test podman configuration * restore changes to testing (removes podman discussed in nf-core#2675) * restore changes to other modules (removes cellranger discussed in nf-core#2646) * update podman settings in UniverSC docs * update podman parameters * update container version for universc to stable release 1.2.5 * remove conda tests for universc (not supported) * update container version for universc to latest release 1.2.5.1 (run tests on pushed version on personal account) * update container version for universc to use nfcore/universc:1.2.5.1 mirror * exit logic for universc module that doesn't support conda consistent with other modules exit logic for modules that dont support conda nf-core#2657 * trigger GitHub Actions test for tomkellygenetics/universc:1.2.5.1 * add log files to universc output directory (confirm running subroutines as expected) * correct UniverSC test configuration addresses singularity test issue https://github.com/nf-core/modules/actions/runs/3955706571/jobs/6774566021 * update configuration for singularity in universc tests * test running universc with singularity --fakeroot requires shadow-uidmap::newuidmap installed * update configuration for singularity in universc tests * debug GH Actions configuration for singularity in universc tests * test running singularity with —fakeroot write permissions * test singularity— * revert changes to singularity tests disables singularity for universc (image too large) * update container settings for universc allows running rootless podman or singularity using --runtime crun or --writable-tmpfs apptainer/singularity#3220 * test universc with singularity --writable-tmpfs * revert changes to singularity tests (--writable-tmpfs not supported on GH Actions) * update container settings for universc to call nfcore/universc:1.2.5.1 (pending mirrored version available) * update version in UniverSC citation --------- Co-authored-by: Simon Thomas Kelly <[email protected]> Co-authored-by: Gisela Gabernet <[email protected]> Co-authored-by: TomKellyGenetics <tomkellygenetics@gmail> Co-authored-by: Alexander Peltzer <[email protected]>
* initialise template for new module: "universc" * update UniverSC module updates metadata, docker container source, licensing, and citations * update UniverSC module unit tests and documentation * update UniverSC module add inputs, outputs and example calls for UniverSC and Cell Ranger v3.0.2 calls versions for UniverSC and Cell Ranger * initialise template for new module: "universc" * update UniverSC module updates metadata, docker container source, licensing, and citations * update UniverSC module unit tests and documentation * update UniverSC module add inputs, outputs and example calls for UniverSC and Cell Ranger v3.0.2 calls versions for UniverSC and Cell Ranger * resolve formatting issues for UniverSC module * resolve linting errors for UniverSC module * fix test jobs to call UniverSC version without errors * correct configuration for UniverSC test jobs * correct linting errors for UniverSC module * correct docker build files for UniverSC * correct syntax errors in cellranger version call * prettier docs for UniverSC * add output to test data for UniverSC module * update UniverSC module to restructured repo nf-core/modules#2141 remove files for restructed UniverSC module (avoids duplicate tests) * define separate outputs for Cell Ranger and UniverSC tests * remove TODO statements and update UniverSC meta.yml defines input and output variables and triggers automated tests * resolve minor linting issues with UniverSC * update paths to UniverSC module in test config * update container versions and tags for UniverSC module * simplifiy container configurations for UniverSC * update configuration for UniverSC tests (build Cell Ranger transcriptome reference first) * test UniverSC module with Cell Ranger references * update tests for UniverSC for restructured repository * update reference inputs for UniverSC module * set up references for cellranger OS test * resolve permissions errors for starting UniverSC * update input arguments for UniverSC and Cell Ranger OS (tests passing) * correct versions and checksums for UniverSC tests * resolves linting issues for UniverSC * resolves linting issues for UniverSC * update expected test outputs for UniverSC tests * migrate UniverSC tests to calling open source Cell Ranger uses Cell Ranger 3.0.2 OS implementation (MIT License) tests passing locally * migrate changes to source code to updated UniverSC container * update unit tests for UniverSC to correct output (using new container) * update output criteria for UniverSC unit tests * change output directory for UniverSC unit tests * test adding podman to GitHub actions (will revert if reviewers object to it) * correct test errors for Cell Ranger OS tests (UniverSC module) * array format for test checks (UniverSC and Cell Ranger OS) * remove unncessary files from UniverSC module * remove podman from automated testing * remove mentions of nf-core/universc container * call executable script from PATH in UniverSC container * migrate UniverSC module Cell Ranger OS count own directory * reorganise UniverSC submodules * update process names in UniverSC module for consistency * update formatting for UniverSC tests * update unit tests for UniverSC submodules running Cell Ranger OS 3.0.2 * reorganise UniverSC submodules to fit naming conventions * remove stub from UniverSC for testing * add universc/mkfastq to unit tests * correct syntax in cellranger module files * update expected output for universc and universc/count tests now consistent with cellranger/count module * correct syntax for universc/count meta.yaml (pass linting) * add stub to universc and universc/count * update unit tests for universc module * update unit tests for universc module * update unit tests for universc module * update unit tests for universc module * update universc/mkfastq test to run stub * update universc/mkfastq expected outputs when running stub * remove trailing whitespace (linting error) * restructure UniverSC main module * update configuration to run each UniverSC test once only * correct UniverSC unit test configuration * update name of tools in universc tests configuration * update expected output for Cell Ranger and UniverSC tests * updates unit tests for Cell Ranger and UniverSC uses contain for web summary HTML as suggested by @apeltzer nf-core/modules#1706 * updates unit tests for Cell Ranger for web summary HTML * updates unit tests for Cell Ranger for web summary HTML * updates unit tests for Cell Ranger for web summary HTML to use description * correct path to Cell Ranger test output * update container options for run UniverSC with singularity runs without root priviledges in writeable container * migrate UniverSC container to mirrored image at nfcore/universc:1.2.4 adds documentation for image build configuration discussed in nf-core/modules#1706 * remove redundant submodules from UniverSC with functions already supported by Cell Ranger module nf-core/modules#1706 * migrate UniverSC references to generate by Cell Ranger submodules * update test configuration for universc/launch * update expected outputs for UniverSC to use Cell Ranger references * update expected outputs for UniverSC to use Cell Ranger references * remove UniverSC submodules for mkref and mkgtf (already implement in Cell Ranger module) discussed in nf-core/modules#1706 * move universc/launch submodule to universc module * remove tests for UniverSC submodules for mkref and mkgtf (already implement in Cell Ranger module) discussed in nf-core/modules#1706 * move tests for universc/launch submodule to universc module * migrate universc/launch submodule to universc module * update paths in unit tests from universc/launch to universc * update documentation for UniverSC module * update paths in test config from universc/launch to universc * restore cellranger module (remove changes from PR 1706) * restore cellranger module (remove changes from PR 1706) * restore cellranger module (remove changes from PR 1706) * update style of documentation to pass linting * add podman to settings and docs (passes local test) * test podman configuration * test podman configuration * restore changes to testing (removes podman discussed in nf-core/modules#2675) * restore changes to other modules (removes cellranger discussed in nf-core/modules#2646) * update podman settings in UniverSC docs * update podman parameters * update container version for universc to stable release 1.2.5 * remove conda tests for universc (not supported) * update container version for universc to latest release 1.2.5.1 (run tests on pushed version on personal account) * update container version for universc to use nfcore/universc:1.2.5.1 mirror * exit logic for universc module that doesn't support conda consistent with other modules exit logic for modules that dont support conda nf-core/modules#2657 * trigger GitHub Actions test for tomkellygenetics/universc:1.2.5.1 * add log files to universc output directory (confirm running subroutines as expected) * correct UniverSC test configuration addresses singularity test issue https://github.com/nf-core/modules/actions/runs/3955706571/jobs/6774566021 * update configuration for singularity in universc tests * test running universc with singularity --fakeroot requires shadow-uidmap::newuidmap installed * update configuration for singularity in universc tests * debug GH Actions configuration for singularity in universc tests * test running singularity with —fakeroot write permissions * test singularity— * revert changes to singularity tests disables singularity for universc (image too large) * update container settings for universc allows running rootless podman or singularity using --runtime crun or --writable-tmpfs apptainer/singularity#3220 * test universc with singularity --writable-tmpfs * revert changes to singularity tests (--writable-tmpfs not supported on GH Actions) * update container settings for universc to call nfcore/universc:1.2.5.1 (pending mirrored version available) * update version in UniverSC citation --------- Co-authored-by: Simon Thomas Kelly <[email protected]> Co-authored-by: Gisela Gabernet <[email protected]> Co-authored-by: TomKellyGenetics <tomkellygenetics@gmail> Co-authored-by: Alexander Peltzer <[email protected]>
See discussion on nf-core Slack