You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
a wrapper object npg_pipeline::product to represent a product
use products attribute to drive p4_stage1, seq_alignment and autoqc
create composition.json files to guide archiving
p4 params files for seq_alignment moved from no_cal/laneN to no_cal
(changes run folder structure when merging lanes)
cluster_count and seqchksum_comparator checks now done at run level instead
of lane level
upfront definition of all products
generic runfolder scaffolding for any products
since the top-level qc directory is no longer required, the tileviz
directory is moved to the analysis directory
reshuffle of roles in npg_pipeline::roles:
npg_pipeline::roles::business::base merged into npg_pipeline::base;
npg_pipeline::roles::business::flag_options moved to
npg_pipeline::base::options, a number of pipeline options from other
modules moved to this role;
npg_pipeline::roles::accessors moved to npg_pipeline::base::config;
helper functions moved to a new role - npg_pipeline::function::util
ref_adapter_pre_exec_string method renamed to repos_pre_exec_string
metadata_cache_dir method, formerly in npg_pipeline::roles::business::base,
removed; npg_pipeline::function::p4_stage1_analysis module, the only user
of this function, switched to use the relevant accessor from the
npg_pipeline::runfolder_scaffold role
minor changes for bcfstats qc check
executor type (lsf or wr) can be specified in the configuration file
wr executor:
set per-job priority;
increase priority for p4 stage 1 job and its predecessors;
set priority of status and start-stop jobs to zero so that
they are executed immediately, but still within dependencies
and memory constraints;
map queues to arbitrary wr options, in particular, a special queue
for p4_stage1 maps to a specific cloud host flavour
correction of build method for rpt_list attribute in product
make bam_cluster_count_check pipeline job dependent on
qc_spatial_filter (in function_list_central.json)
archival daemon - limit number of simultaneously archived NovaSeq runs
wr executor - explicitly propagate pipeline's environment to jobs
illumina archiver job:
exclude discontinued verbose attribute and paths that are not needed
for the minimal work this loader is doing now;
remove LSF preexec requesting that the job is a unique runner since
db queries are much simpler now
change signature of the autoqc archival job in line with extended
functionality of the autoqc db loader (ability to find JSON files
in the run folder)
change components_as_products method of npg_pipeline::product to
return a list with one item when there is only one component in
the composition (instead of an empty list)
tileviz index file with links to lane-level tileviz reports is created
seq_alignment supports HISAT2 aligner for RNA libraries
explicit iRODS destination collection is set for iRODS loaders,
/seq/illumina/runs/RUN_ID for NovaSeq runs and /seq/RUN_ID
for the rest
explicitly use iRODS loader from an 'old' dated directory for
old style runfolders
a new function, archive_run_data_to_irods, to publish run-level non-product data to iRODS
modify run_data_to_irods_archiver module to ensure the interop files go to a dedicated directory