PopPUNK v2.6.0

Main changes:

• Lineage fits now use reciprocal best match with --reciprocal-only, --count-unique-distances and --max-search-depth, which gives better results.
• Fixes for threshold model assignment

What's Changed

• Update parsing fixes branch with new master by @nickjcroucher in #195
• Update with recent changes to master by @nickjcroucher in #205
• Integrate changes from V250 candidate by @nickjcroucher in #206
• Small fixes for beebop by @muppi1993 in #217
• Update poppunk_iterate.py by @BZhao95 in #216
• Read the docs sphinx version by @johnlees in #215
• Adds poppunk_distribute_fit.py by @samhorsfield96 in #226
• Update with changes to master by @nickjcroucher in #230
• Patch for relative paths in plot fit by @johnlees in #236
• Fix option names for k-mer range min/max and k-mer step. by @tmaklin in #239
• Fix for assign with threshold models by @johnlees in #240
• Lineage model fitting - PopPUNK changes by @nickjcroucher in #232
• Remove blas by @johnlees in #244
• Proceed with update-db on QC failure by @johnlees in #245

New Contributors

• @BZhao95 made their first contribution in #216
• @samhorsfield96 made their first contribution in #226
• @tmaklin made their first contribution in #239

Full Changelog: v2.5.0...v2.6.0

PopPUNK v2.5.0

Minimum sketchlib version for this release is v2.0.0

New features:

• Dendropy replaced with faster & more reliable alternatives #203
• A new logo #202
• Improve iterative PopPUNK code
• Documentation update and improvements #191
• Deal better with name clash when querying #190
• Make manual start a bit easier to use #174
• Replace t-SNE with mandrake
• Output .microreact files, and allow direct creation of Microreact instances with an API key
• Various QC additions to help with multi-cluster merges #194

Bug fixes:

• Various fixes to cytoscape visualisation #185 #196 #210
• Hide progress bars when using --plot-fit
• Stop always checking query-query dists when clustering (and potential bug adding them to network twice)
• Fix N QC when working with reads #207

What's Changed

• Upgrade of GPU refinement by @nickjcroucher in #164
• Remove start_point concept from refine fit by @johnlees in #168
• Upgrades to refinement functions by @nickjcroucher in #175
• Update MST calculation documentation by @nickjcroucher in #177
• Move the extend algorithm into the C++ extension by @johnlees in #178
• Multi-boundary method by @johnlees in #180
• Bacpop 17 by @muppi1993 in #201
• Add rapidnj to docker image by @muppi1993 in #212
• Release for v2.5.0 by @johnlees in #204
• pip installable poppunk, almost by @richfitz in #209

New Contributors

• @muppi1993 made their first contribution in #201
• @richfitz made their first contribution in #209

Full Changelog: v2.4.0...v2.5.0

PopPUNK v2.4.0

Minimum sketchlib version for this release is v1.7.0

To use --gpu-graph requires cudf and cugraph to be installed from the nvidia conda channel, which is not part of the standard installation)

New features:

• Adds minimum spanning tree computation and visualisation #141 #148
• Add two new network scores based on betweenness #146
• Move boundary code into a C++ extension in this package #146 #158
• Adds GPU accelerated graphs #87 #148
• Adds a docker container which is used for web.poppunk.net #151 #162
• New github actions for testing and building the web API #151
• Add progress bars in for model assignment #155
• Parallelise model assignment #155
• Adds the VLKC terminology, and 'unword' cluster names #161

Bug fixes:

• Correctly specify thread count with rapidnj #139
• Regenerate random match changes after --update-db #149
• Fix issue with label order when using --update-db more than once #152
• Update some scripts/ to work with newer versions of numpy and scikit-learn #160
• Keep hyphens in sample names in trees #159
• Fix a plot name #158
• Pin some package versions #140 #142

PopPUNK v2.3.0

This is a major (API-breaking) update which moves the assign and visualisation functions into their own programs, to make the program more modular. The minimum version of pp-sketchlib required is 1.6.0.

New features:

• Lineage assign mode uses matrix code in pp-sketchlib #108
• New algorithm for clique pruning #110
• Visualisation and query moved out of main, and into their own programs #112 #115 #129
• Simpler CLI defaults #125
• Updated documentation #122
• Add edge weights to graph #123
• Add API for use of poppunk_assign with a http server #124 #131
• Add corrected/uncorrected distances when plotting k-mer fits #136

Bug fixes:

• More stable generation of documentation #132
• Fixes continue mode for QC function #134
• Fixes long length QC fail #137

PopPUNK v2.2.0

The first bug fix will affect many results, and all users are encouraged to upgrade

New features:

• More thorough sample QC using pp-sketchlib features (#101)
• Update to pp-sketchlib v1.5.1 (#104)

Bug fixes:

• Misordered labels with older versions of pp-sketchlib (#95)
• TypeError with visualisations (#99)
• networkx still used in reference prune program (#97)

Sketchlib 1.4.0

• Update to sketchlib 1.4.0 (#90) (https://github.com/johnlees/pp-sketchlib/releases/tag/v1.4.0)
• Use faster, threaded matrix functions (#78, #80)
• Change from networkx to graph-tool (#83)
• Use sharedmem (#76)
• Add --lineage-clustering mode (#72)
• Better --refine-model default boundary (#94)

NB python >=3.8 is now required (#81, #76)

Update sketchlib to v1.3.0

This accommodates updates to pp-sketchlib introduced since v1.1.0, mostly passing through new command line parameters (e.g. --use-gpu). See https://github.com/johnlees/pp-sketchlib/releases/tag/v1.3.0 for more info.

New sketchlib backend

This is a major new release of PopPUNK, which uses a new 'backend' to do sketching and distance calculation, pp-sketchlib

This changes the input format and the API somewhat, will be incompatible with previous database versions, and generates slightly different distance results. If you need backwards compatibility the previous version can still be run by specifying --use-mash.

New features:

• Use pp-sketchlib as the backend. This is ~2x faster for sketching and 50-100x faster for distance calculations. Databases are ~1/4 of the size.
• Input data is now formatted as a tab separated file with name followed by any associated sequence. Sample names no longer have to be filenames. (Closes #43, #46)
• Read data can now be handled, including a filter to remove k-mers containing sequencing errors
• Faster database edits with prune_db and reference_pick
• Ability to use the previous sketching method and databases by specifying --use-mash

Bug fixes:

• Better error handling when creating visualisations, so output files are still produced if this fails

Add --threshold option

This adds:

• The ability to use a simple core threshold (#49)
• Use external clusters in generate viz mode (#55)

Other:

• Add scripts into package (#56)

Better reference selection

The main reason for this release is the fix in #50. Although not necessarily a problem, in some populations the reference picking step can lead the pruned network to appear inconsistent with the clusters, giving warning messages when assigning new queries. This is fixed here.

Bug fixes:

• Fixes to --generate-viz mode (#44).
• Problems using relative paths (#48).
• Ensuring consistent network and cluster definitions when choosing references (#50).

Other:

• Update travis-CI to use correct DISPLAY for matplotlib
• Add Azure pipelines CI