PALEOMIX v1.2.8

This is minor release of PALEOMIX, which includes an important bug-fix to BAM Pipelines using the BWA
'mem' or 'bwasw' algorithms. Previously, user-specified command-line parameters would not be correctly
applied to the BWA commands, when either of these two algorithms were used. This has now been
corrected.

Added

• Added FILTER entry for 'F' filter used in vcf_filter. This corresponds to
  heterozygous sites where the allele frequency was not determined.
• Added 'dupcheck' command. This command roughly corresponds to the
  DetectInputDuplication step that is part of the BAM pipeline, and attempts
  to identify duplicate data (not PCR duplicates), by locating reads mapped
  to the same position, with the same name, sequence, and quality scores.
• Added link to sample data used in publication to the Zonkey documentation.

Changed

• Only letters, numbers, and '-', '_', and '.' are allowed in sample-names
  used in Zonkey, in order to prevent invalid filenames and certain programs
  breaking on whitespace. Trailing whitespace is stripped.
• Show more verbose output when building Zonkey pipelines.
• Picard tools version 1.137 or later is now required by the BAM pipeline.
  This is nessesary as newer BAM files (header version 1.5) would fail to
  validate when using earlier versions of Picard tools.

Fixed

• Fixed validation nodes failing on output paths without a directory.
• Fixed possible uncaught exceptions when terminating cat commands used by
  FASTQ validation nodes resulting in loss of error messages.
• Fixed makefile validation failing with an unhandled TypeError if unhashable
  types were found in unexpected locations. For example, a dict found where a
  subset of strings were allowed. These now result in a proper MakeFileError.
• Fixed user options in the 'BWA' section of the BAM Pipeline makefiles not
  being correctly applied when using the 'mem' or the 'bwasw' algorithms.
• Fixed some unit tests failing when the environment caused getlogin to fail.

PALEOMIX v1.2.7

Added

• PALEOMIX now includes the 'Zonkey' pipeline, a pipeline for
  detecting equine F1 hybrids from archeological remains. Usage
  is described in the documentation.

Changed

• The wrongly named per-sample option 'Gender' in the phylogenetic
  pipeline makefile has been replaced with a 'Sex' option. This does not
  break backwards compatibility, and makefiles using the old name will still
  work correctly.
• The 'RescaleQualities' option has been merged with the 'mapDamage' Feature
  in the BAM pipeline makefile. The 'mapDamage' feature now takes the options
  'plot', 'model', and 'rescale', allowing more fine-grained control.

Fixed

• Fixed the phylogenetic pipeline complaining about missing sample genders
  (now sex) if no regions of interest had been specified. The pipeline will
  now complain about there being no regions of interest, instead.
• The 'random sampling' genotyper would misinterpret mapping qualities 10
  (encoded as '+') and 12 (encoded as '-') as indels, resulting in the
  genotyping failing. These mapping qualities are now correctly ignored.

Minor revision

Changed

• PALEOMIX now uses the 'setproctitle' for better compatibility; installing
  / upgraing PALEOMIX using pip (or equivalent tools) should automatically
  install this dependency.

Fixed

• mapDamage plots should not require indexed BAMs; this fixed missing file
  errors for some makefile configurations.
• Version check for java did now works correctly for OpenJDK JVMs.
• Pressing 'l' or 'L' to list the currently running tasks now correctly
  reports the total runtime of the pipeline, rather than 0s.
• Fixed broken version-check in setup.py breaking on versions of python
  older than than 2.7, preventing meaningful message (patch by beeso018).
• The total runtime is now correctly reported when pressing the 'l' key
  during execution of a pipeline.
• The logger will automatically create the output directory if this does
  not already exist; previously logged messages could cause the pipeline
  to fail, even if these were not in themselves fatal.
• Executables required executables for version checks are now included in
  the prior checks for missing executables, to avoid version-checks failing
  due to missing executables.

Added

• PALEOMIX will attempt to automatically limit the per-process maximum
  number of file-handles used by when invoking Picard tools, in order
  to prevent failures due to exceeding the system limits (ulimit -n).

PALEOMIX v1.2.5

Minor revision:

Changed

• Improved information capture when a node raises an unexpected exception,
  mainly for nodes implementing their own 'run' function (not CommandNodes).
• Improved printing of the state of output files when using the command-line
  option --list-output-files. Outdated files are now always listed as
  outdated, where previously these could be listed as 'Missing' if the task
  in question was queued to be run next.
• Don't attempt to validate prefixes when running 'trim_pipeline'; note that
  the structure of the Prefix section the makefile still has to be valid.
• Reverted commit normalizing the strand of unmapped reads.
• The commands 'paleomix coverage' and 'paleomix depths' now accept records
  lacking read-group information by default; these are record as in the
  sample and library columns. It is further possible to ignore all read-group
  information using the --ignore-readgroups command-line option.
• The 'bam_pipeline mkfile' command now does limited validation of input
  'SampleSheet.csv', prints generated targets sorted alphabetically, and
  automatically generates unique names for identically named lanes. Finally,
  the target template is not included automatically generating a makefile.
• The 'coverage' and 'depth' commands are now capable of processing files
  containing reads with and without read-groups, without requiring the use
  of the --ignore-readgroups command-line option. Furthermore, reads for
  which the read-group is missing in the BAM header are treated as if no
  readgroup was specified for that read.
• The 'coverage' and 'depth' command now checks that input BAM files are
  sorted during startup and while processing a file.
• Normalized information printed by different progress UIs (--progress-ui),
  and included the maximum number of threads allowed.
• Restructured CHANGELOG based on http://keepachangelog.com/

Fixed

• Fixed mislabeling of BWA nodes; all were labeled as 'SE'.
• Terminate read duplication checks when reaching the trailing, unmapped
  reads; this fixes uncontrolled memory growth when an alignment produces a
  large number of unmapped reads.
• Fixed the pipeline demanding the existence of files from lanes that had
  been entirely excluded due to ExcludeReads settings.
• Fixed some tasks needlessly depending on BAM files being indexed (e.g.
  depth histograms of a single BAM), resulting in missing file errors for
  certain makefile configurations.
• Fixed per-prefix scan for duplicate input data not being run if no BAMs
  were set to be generated in the makefile, i.e. if both 'RawBAM' and
  'RealignedBAM' was set to 'off'.

Deprecated

• Removed the BAM file from the bam_pipeline example, and added deprecation
  warning; support for including preexisting BAMs will be removed in a
  future version of PALEOMIX.

PALEOMIX v1.2.4

Minor revision:

• Fix regression causing 'fixmate' not to be run on paired-end reads. This would occasionally cause paired-end mapping to fail during validation.
• Include PATH in 'pipe.errors' file, to assist debugging of failed nodes.

PALEOMIX v1.2.3

Minor revision:

2016-03-11: Added the ability to the pipelines to output the list of input
          files required for a given makefile, excluding any file built
          by the pipeline itself. Use the --list-output-files command-
          line option to view these.
2016-03-11: Updated 'bam_pipeline' makefile template; prefixes and targets
          are described more explicitly, and values for the prefix are
          commented out by default. The 'Label' option is no included in
          the template, as it is considered deprecated.
2016-03-11: Allow the 'trim_pipeline' to be run on a makefile without any
          prefixes; this eases use of this pipeline in the case where a
          latter mapping is not wanted.
2016-03-11: Improved handling of unmapped reads in 'paleomix cleanup';
          additional flags (in particular 0x2; proper alignment) are now
          cleared if the mate is unmapped, and unmapped reads are always
          represented on the positive strand (clearing 0x4 and / or 0x20).