Releases: wtsi-npg/npg_seq_pipeline
Releases · wtsi-npg/npg_seq_pipeline
62.0.2
57.5.0
- add BWA MEM2 support to seq_alignment function
- bug fix: add -f to rm command removing intermediate files (to avoid error when no intermediate files are present)
- allow selection of duplicate marking method (biobambam,samtools or picard) in seq_alignment via product_release.yml
- detect flowcell type and set uses_patterned_flowcell attribute to allow setting of optical duplicate region size
- add ability to select bwakit postalt processing (if reference has alternate haplotypes) in seq_alignment via product_release.yml
56.0.0
- add autosome target to seq_alignment
- pipeline configuration module and product release configuration accessors are moved to npg_tracking package in order for the product configuration be accessible from other packages, code in this package refactored to accommodate the change
- conform to bambi's v 0.12.0 file and directory naming schema for tileviz data
- add facility to do LSF 1:1 job index dependencies on array jobs
- when validating run folder for deletion, ensure linked directories and files are recognised
release 53.0
- a wrapper object npg_pipeline::product to represent a product
- use products attribute to drive p4_stage1, seq_alignment and autoqc
- create composition.json files to guide archiving
- p4 params files for seq_alignment moved from no_cal/laneN to no_cal
(changes run folder structure when merging lanes) - cluster_count and seqchksum_comparator checks now done at run level instead
of lane level - upfront definition of all products
- generic runfolder scaffolding for any products
- since the top-level qc directory is no longer required, the tileviz
directory is moved to the analysis directory - reshuffle of roles in npg_pipeline::roles:
npg_pipeline::roles::business::base merged into npg_pipeline::base;
npg_pipeline::roles::business::flag_options moved to
npg_pipeline::base::options, a number of pipeline options from other
modules moved to this role;
npg_pipeline::roles::accessors moved to npg_pipeline::base::config;
helper functions moved to a new role - npg_pipeline::function::util - ref_adapter_pre_exec_string method renamed to repos_pre_exec_string
- metadata_cache_dir method, formerly in npg_pipeline::roles::business::base,
removed; npg_pipeline::function::p4_stage1_analysis module, the only user
of this function, switched to use the relevant accessor from the
npg_pipeline::runfolder_scaffold role - minor changes for bcfstats qc check
- executor type (lsf or wr) can be specified in the configuration file
- wr executor:
set per-job priority;
increase priority for p4 stage 1 job and its predecessors;
set priority of status and start-stop jobs to zero so that
they are executed immediately, but still within dependencies
and memory constraints;
map queues to arbitrary wr options, in particular, a special queue
for p4_stage1 maps to a specific cloud host flavour - correction of build method for rpt_list attribute in product
- make bam_cluster_count_check pipeline job dependent on
qc_spatial_filter (in function_list_central.json) - archival daemon - limit number of simultaneously archived NovaSeq runs
- wr executor - explicitly propagate pipeline's environment to jobs
- illumina archiver job:
exclude discontinued verbose attribute and paths that are not needed
for the minimal work this loader is doing now;
remove LSF preexec requesting that the job is a unique runner since
db queries are much simpler now - change signature of the autoqc archival job in line with extended
functionality of the autoqc db loader (ability to find JSON files
in the run folder) - change components_as_products method of npg_pipeline::product to
return a list with one item when there is only one component in
the composition (instead of an empty list) - tileviz index file with links to lane-level tileviz reports is created
- seq_alignment supports HISAT2 aligner for RNA libraries
- explicit iRODS destination collection is set for iRODS loaders,
/seq/illumina/runs/RUN_ID for NovaSeq runs and /seq/RUN_ID
for the rest - explicitly use iRODS loader from an 'old' dated directory for
old style runfolders - a new function, archive_run_data_to_irods, to publish run-level non-product data to iRODS
- modify run_data_to_irods_archiver module to ensure the interop files go to a dedicated directory
- additional tags for NovaSeq in dbic_fixtures