'Le Big' Main Winter Cleanup

[martenson]

requires installation/uninstallation

• install datamash tools into their own section (reverse, transpose, datamash)
• install trimmomatic, trim galore, pear
• update BWA and MEM
• RNA analysis to MAIN except of trinity
• sam tools merge to MAIN except call and phase
• NVC, freebayes, varian annotator latest to MAIN
• variant analysis merge into MAIN without GEMINI
• chipseq to MAIN + bamfilter
• bedtools to MAIN
• statistics merge into MAIN
• multivariate analysis section mirror test to MAIN
• multiple regression section mirror test to MAIN
• graph section: merge into MAIN
• install str_fm microsat analysis (https://github.com/Arkarachai/STR-FM)
• metagenomics: skip diamond, propagate rest, deprecate what is on Main,
• NGS: DuNovo propagate to MAIN (resource selector ping @natefoo )
• add MEME to motif section (Add wrapper for meme version 4.11.0 #478)
• MAFFT to multiple alignment section
• install HISAT2 on MAIN

tooldev related

• fix help on sailfish fix sailfish help bgruening/galaxytools#283
• test HISAT2 @nekrut
• fix heatmap tool images update (and adopt?) heatmap tool #495

org related

• mutate codons to variant analysis
• variant analysis out of BETA
• move extract feature section to operate on gen intervals
• combine fetch section into one
• ask about sed/awk security issues (@natefoo)
• tab to fastq and fastq to tab move to convert section from NGS QC (there are more like this, review)
• histogram move to graph display data (remove form filter)
• get genomic scores move to operate on genomic intervals section
• remove ngs toolbox beta
• introduce DEPRECATED section
• make old picard disappear
• make GATK deprecated
• section CloudMap and move CloudMap tools there
• group all NGS sections next to each other
• remove RNA STRUCTURE PREDICTION label
• hide old snpeff tools

DM installed on Test

• HISAT2
• bowtie
• bowtie2

reference data and indexes

tracked at https://github.com/galaxyproject/galaxy/issues/1470

problems

• sailfish cannot install its dependency, cmake (timeout)
• unable to install deeptools (even on Test)
• other sailfish issues sailfish issues bgruening/galaxytools#284
• update bamtools - needs some DB fixing (remove the need of saving path to config files in the TS database galaxy#1390)
• sailfish DM has incorrect dependencies
• sailfish indexes (see above)

[nekrut]

GEMINI should follow shortly once it works on test for @bgruening and me (it may already work: I just did not try yet)

[bgruening]

• @natefoo @martenson regading security: https://github.com/bgruening/galaxytools/tree/master/tools/text_processing/text_processing#note-about-security
• @deeptools 2.0 will be released this year
• HISAT2 not HISAT, we should deprecate HISAT
  @nekrut working on a new gemini release: https://github.com/galaxyproject/tools-iuc/tree/gemini but the old one is working for us here
• @nekrut diamond not good enough (just interested, I don't have a string feeling here)

[natefoo]

I concur with @bgruening regarding the security.

@nekrut, which resource selector did you want on Main? Right now the tools that can go to Stampede get a selector for that (and also for the "test/development" queue).

[martenson]

thanks @natefoo @bgruening !

[natefoo]

'Le Grand'?

[martenson]

too French

[martenson]

This is mostly done, see the checklist.

[bgruening]

This is amazing! usegalaxy.org will rock even harder! Thanks @martenson!

[jennaj]

HISAT2 DM has a problem with advanced options re SNP annotation. Wrapper needs update? See: unrecognized arguments

Fatal error: Exit code 2 ()
usage: extract_snps.py [-h] [-v] [--testset] [genome_file] [snp_file]
extract_snps.py: error: unrecognized arguments: --genome_file --snp_file snps.tabular

[martenson]

@jennaj please create a new issue for that

[jennaj]

@martenson Done, thanks! #530

[martenson]

I added bunch of labels to new and updated tools on Main. I probably missed some. Let me know if something is missing/wrong.

Closing as other issues are not cleanup-related and have their own cards.