Running the pipeline with singularity

[ramakrishnas]

I had success running the test but when I tried to run the pipeline with my data it is not working and gives the message
-[nf-core/mag] Pipeline completed with errors-

If some can take a look at my command below and let me know where I'm doing wrong that will be helpful.

nextflow run nf-core/mag -r 1.1.0 --input '/Users/rsompallae/*_R{1,2}.fastq' -profile singularity -with-singularity /Users/rsompallae/biotools/nfcore-mag-1.1.0.sif -c nfcore_ui_hpc_2.config --busco_reference /Users/rsompallae/biotools/bacteria_odb10.2020-03-06.tar.gz

Thanks
Rama

[d4straub]

Hi, glad you got mag running.
It would be extremely helpful to have a little more information, i.e. the error message. What did the pipeline complain about?

[ramakrishnas]

based on the log file (.nextflow.log) I see following error.

Nov-16 13:46:24.338 [main] WARN nextflow.script.ScriptBinding - Access to undefined parameter input_paths -- Initialise it to a default value eg. params.input_paths = some_value
Nov-16 13:46:24.342 [main] WARN nextflow.script.ScriptBinding - Access to undefined parameter reads -- Initialise it to a default value eg. params.reads = some_value
Nov-16 13:46:24.429 [PathVisitor-1] ERROR nextflow.Channel - null
java.lang.NullPointerException: null
at nextflow.file.FileHelper.isGlobAllowed(FileHelper.groovy:233)
at nextflow.file.PathVisitor.applyGlobPattern0(PathVisitor.groovy:91)
at nextflow.file.PathVisitor.apply(PathVisitor.groovy:71)
at nextflow.file.PathVisitor$_applyAsync_closure1.doCall(PathVisitor.groovy:77)
at nextflow.file.PathVisitor$_applyAsync_closure1.call(PathVisitor.groovy)
at groovy.lang.Closure.run(Closure.java:486)
at java.util.concurrent.CompletableFuture.uniRun(CompletableFuture.java:719)
at java.util.concurrent.CompletableFuture$UniRun.tryFire(CompletableFuture.java:701)
at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:456)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Nov-16 13:46:24.437 [PathVisitor-1] DEBUG nextflow.Session - Session aborted -- Cause: java.lang.NullPointerException

[d4straub]

The line Nov-16 13:46:24.342 [main] WARN nextflow.script.ScriptBinding - Access to undefined parameter reads -- Initialise it to a default value eg. params.reads = some_value indicates that the parameter reads, i.e. --reads, is expected, but not given. In mag v1.0.0 --reads was the typical input, however, in v1.1.0 and onward, it is --input. This leads to the conclusion that you are not actually running v1.1.0.
Additionally, -profile singularity -with-singularity <container> is redundant. Use just one of those. -profile singularity will automatically download the correct container, -with-singularity <container> will use a local container. I'd recommend -profile singularity. If you do want to use -with-singularity <container>, first do singularity pull docker://nfcore/mag:<version>. A list of containers can be found here.

Please test the following:

nextflow pull nf-core/mag -r 1.1.1
nextflow run nf-core/mag -r 1.1.1 --input '/Users/rsompallae/*_R{1,2}.fastq' -profile singularity -c nfcore_ui_hpc_2.config --busco_reference /Users/rsompallae/biotools/bacteria_odb10.2020-03-06.tar.gz

[ramakrishnas]

Thank you Daniel for your kind help.

I followed your instructions, pulled v1.1.1 and tried but seeing the same error

nextflow run nf-core/mag -r 1.1.1 --input '/Users/rsompallae/*_R{1,2}.fastq' -profile singularity -c nfcore_ui_hpc_2.config --busco_reference /Users/rsompallae/biotools/bacteria_odb10.2020-03-06.tar.gz
N E X T F L O W ~ version 19.10.0
Launching nf-core/mag [agitated_crick] - revision: a3122b1 [1.1.1]
WARN: Access to undefined parameter input_paths -- Initialise it to a default value eg. params.input_paths = some_value
WARN: Access to undefined parameter reads -- Initialise it to a default value eg. params.reads = some_value
Unexpected error [NullPointerException]

Here is the text from the log file

Nov-17 19:19:53.166 [main] DEBUG nextflow.Session - Work-dir: /Users/rsompallae/work [nfs]
Nov-17 19:19:53.268 [main] DEBUG nextflow.Session - Observer factory: TowerFactory
Nov-17 19:19:53.349 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
Nov-17 19:19:54.083 [main] DEBUG nextflow.Session - Session start invoked
Nov-17 19:19:54.088 [main] DEBUG nextflow.trace.TraceFileObserver - Flow starting -- trace file: /Users/rsompallae/results/pipeline_info/execution_trace.txt
Nov-17 19:19:58.060 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
Nov-17 19:19:58.551 [main] WARN nextflow.script.ScriptBinding - Access to undefined parameter input_paths -- Initialise it to a default value eg. params.input_paths = some_value
Nov-17 19:19:58.555 [main] WARN nextflow.script.ScriptBinding - Access to undefined parameter reads -- Initialise it to a default value eg. params.reads = some_value
Nov-17 19:19:58.560 [PathVisitor-1] ERROR nextflow.Channel - null
java.lang.NullPointerException: null
at nextflow.file.FileHelper.isGlobAllowed(FileHelper.groovy:233)
at nextflow.file.PathVisitor.applyGlobPattern0(PathVisitor.groovy:91)
at nextflow.file.PathVisitor.apply(PathVisitor.groovy:71)
at nextflow.file.PathVisitor$_applyAsync_closure1.doCall(PathVisitor.groovy:77)
at nextflow.file.PathVisitor$_applyAsync_closure1.call(PathVisitor.groovy)
at groovy.lang.Closure.run(Closure.java:486)
at java.util.concurrent.CompletableFuture.uniRun(CompletableFuture.java:719)
at java.util.concurrent.CompletableFuture$UniRun.tryFire(CompletableFuture.java:701)
at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:456)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Nov-17 19:19:58.650 [PathVisitor-1] DEBUG nextflow.Session - Session aborted -- Cause: java.lang.NullPointerException

[skrakau]

Hi @ramakrishnas , thanks for reporting this!
This seems like a bug concerning the --input parameter: internally it still processes the params.reads. Something went wrong when I merged the nf-core template 1.10.1. (Although I get a different error message)
You could use the --input_paths parameter (not documented in the help message) as in conf/test.config or the --reads parameter, but then the summary output does not contain this parameter (as it uses already params.input).
Or just wait a bit if you can, I will fix this as soon as possible in the dev branch and prepare a minor release with a patch.

[ramakrishnas]

Thank you so much, I really appreciate it.

…

On Wed, Nov 18, 2020 at 3:59 AM Sabrina Krakau ***@***.***> wrote: Hi @ramakrishnas <https://github.com/ramakrishnas> , thanks for reporting this! This seems like a bug concerning the --input parameter: internally it still processes the params.reads. Something went wrong when I merged the nf-core template 1.10.1. (Although I get a different error message) You could use the --input_paths parameter (not documented in the help message) as in conf/test.config or the --reads parameter, but then the summary output does not contain this parameter (as it uses already params.input). Or just wait a bit if you can, I will fix this as soon as possible in the dev branch and prepare a minor release with a patch. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#131 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGYS55P7ARQR6GDIQIYXYLSQOLGVANCNFSM4TT7VLQA> .

[ramakrishnas]

Hi skrakau, I just want to checkin and see if it is okay for me to pull the version 1.1.2 and test? Thanks Rama

…

On Tue, Nov 24, 2020 at 12:00 PM Sabrina Krakau ***@***.***> wrote: Closed #131 <#131>. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#131 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGYS5ZX7HPH34ZM27RCKQ3SRPYDLANCNFSM4TT7VLQA> .

[skrakau]

Hi @ramakrishnas , yes that would be great! Let us know if it solves your problems.

[ramakrishnas]

Hi, I pulled the version 1.1.2 and tried to run but getting the error 'No such variable: projectDir'. is this something I need to provide with the command options? Thanks

…

On Wed, Nov 25, 2020, 2:42 AM Sabrina Krakau ***@***.***> wrote: Hi @ramakrishnas <https://github.com/ramakrishnas> , yes that would be great! Let us know if it solves your problems. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#131 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGYS55H5TOBYPUOZKL27M3SRS7PNANCNFSM4TT7VLQA> .

[d4straub]

@ramakrishnas Please continue in #138, since this is a different topic!