Bowtie overcommits CPU ressources #5983
Comments
bernt-matthias
added a commit
to bernt-matthias/tools-iuc
that referenced
this issue
May 3, 2024
fixes galaxyproject#5983 by using a pipe bowtie2 and samtools run in parallel. since already bowtie2 alone uses more CPU than the assigned ones (less that 1 core) we should not do this, but run them serially. also properly redirects stderr.
bernt-matthias
added a commit
that referenced
this issue
May 13, 2024
* bowtie2: do not use pipes fixes #5983 by using a pipe bowtie2 and samtools run in parallel. since already bowtie2 alone uses more CPU than the assigned ones (less that 1 core) we should not do this, but run them serially. also properly redirects stderr. * fix linting * remove --no-PG * restore piping * reduce number of used threads by one * Revert "fix linting" This reverts commit 6e1b4f2. * bump * Improve comment wording Co-authored-by: Wolfgang Maier <[email protected]> --------- Co-authored-by: Wolfgang Maier <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
bowtieseems to be extremely efficient, i.e.XGALAXY_SLOTSare used the CPU usage is most of the time larger thanX*100% (if checked intop). This seems to be because the-pparameter seems to control the number of (additional) alignment threads (probably plus 1 control thread).Additionally we pipe the result to
samtoolsusing potentiallyGALAXY_SLOTSmore slots (in practice most of the time less).In summary I would suggest to remove the pipes (to the cost of having an additional temporary file.)
The text was updated successfully, but these errors were encountered: