Missing or corrupted meta map in short reads channel?

[prototaxites]

Description of the bug

Run into the below error a couple of times now on either KRAKEN2 or CENTRIFUGE, which take the short reads channel post-processing as input. The lines they fail at define the prefix, which in all cases should be `${meta.id}$ - at that point, there shouldn't be anything else in the meta map?

The error is sporadic, and I haven't been able to figure out how to trigger it, or what operational sequence of processes have to run to hit it. However, I've only noticed it when testing binning entry, which may or may not be the culprit - as the short read profilers should run sooner here due to skipped preprocessing steps.

-[nf-core/mag] Pipeline completed with errors-
ERROR ~ Error executing process > 'NFCORE_MAG:MAG:CENTRIFUGE (test_minigut-minigut_cf)'

Caused by:
  java.util.NoSuchElementException -- Check script './workflows/../modules/local/centrifuge.nf' at line: 21

Source block:
  def input = meta.single_end ? "-U \"${reads}\"" :  "-1 \"${reads[0]}\" -2 \"${reads[1]}\""
  prefix = task.ext.prefix ?: "${meta.id}"
  """
      centrifuge -x "${db_name}" \
          -p ${task.cpus} \
          --report-file report.txt \
          -S results.txt \
          $input
      centrifuge-kreport -x "${db_name}" results.txt > ${prefix}.centrifuge_kreport.txt
      cat results.txt | cut -f 1,3 > results.krona
  
      cat <<-END_VERSIONS > versions.yml
      "${task.process}":
          centrifuge: \$(centrifuge --version | sed -n 1p | sed 's/^.*centrifuge-class version //')
      END_VERSIONS
      """

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`

 -- Check '.nextflow.log' file for details

#473 (comment)

-[nf-core/mag] Pipeline completed with errors-
ERROR ~ Error executing process > 'NFCORE_MAG:MAG:CENTRIFUGE (test_minigut-minigut_cf)'

Caused by:
  java.util.NoSuchElementException -- Check script './workflows/../modules/local/centrifuge.nf' at line: 21

Source block:
  def input = meta.single_end ? "-U \"${reads}\"" :  "-1 \"${reads[0]}\" -2 \"${reads[1]}\""
  prefix = task.ext.prefix ?: "${meta.id}"
  """
      centrifuge -x "${db_name}" \
          -p ${task.cpus} \
          --report-file report.txt \
          -S results.txt \
          $input
      centrifuge-kreport -x "${db_name}" results.txt > ${prefix}.centrifuge_kreport.txt
      cat results.txt | cut -f 1,3 > results.krona
  
      cat <<-END_VERSIONS > versions.yml
      "${task.process}":
          centrifuge: \$(centrifuge --version | sed -n 1p | sed 's/^.*centrifuge-class version //')
      END_VERSIONS
      """

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`

 -- Check '.nextflow.log' file for details

#470 (comment)

Command used and terminal output

No response

Relevant files

No response

System information

No response

[jfy133]

I think I've identified the problem:

We need to replace all .clone()/.remove() calls with def meta_new = meta - meta.subMap('key')

[jfy133]

Yeah I think this is related: #479

I'm going to go and replace all .clones 😬

thank the lord and saviour of today @robsyme

https://www.youtube.com/watch?v=A357C-ux6Dw&t=879s

[jfy133]

I'm assuming done here: #480

[jfy133]

Please reopen if not