Release for v2.5.0

[johnlees]

Todo:

• Update sketchlib to v2. Use kwargs in all calls.
• Increase recursion depth.
• Replace dendropy Replace dendropy #203
• New logo New logo #202
• Fix the cytoscape output Subsetting the database when using poppunk_visualise --cytoscape with --include-files #196
• Fix tree = None in cytoscape visualisations. combined_seq not set here
• Documentation update Documentation update #191, Update tutorial with new Cytoscape command #185, Update documentation for assign + lineage mode #189
• Reference names Give clearer results when queries are in the reference DB #190
• Tweak manual start Give a 'scaled' option for manual start #174
• Remove vestigal poppunk web code
• Replace t-SNE with mandrake
• Change this line to

  if type(top_value) == str:
      sketch_dict = json.loads(top_value)
  else:
      sketch_dict = top_value
  

• Change to new microreact api, add --api-key option to visualisation
• Output .microreact file
• Update t-SNE -> mandrake images in docs (after microreact updated)
• For cytoscape visualisation add a loop which writes a graphml for each component
• Update assign so high stress/betweenness nodes can be removed (or many zeros, or members of many clusters) anything which links >2 connected components
  • Fix QC distance prune All clusters merged into a single one when trying to assign new strains  #194
  • Possible bug with assignment (4,13,19,53 or 139 in 2.4.0; 4_19_53_139 or 4_13_19 or 4_13_53 in 2.4.6)
  • Add a 'one-by-one' option
• Iterative PopPUNK
  • Distances to treeswift part
  • 'Cut tree' (see fig 1D from Bin)
  • Check for boundaries lower than zero
  • Expand the search range (as an option)
• Run some example data through
  • Lots of distances calculated on create db (this was from plot-fit, now capturing stderr via a complicated method!)
  • Network summaries for lineage fit?
• Stalls on non-pneumo with GPSCs (e.g. 10050_2#1) couldn't replicate
• Potential issue with sketchlib version string HDBSCAN models require consistent version of sklearn to open #213 Fix this line https://github.com/bacpop/PopPUNK/blob/master/PopPUNK/sketchlib.py#L31
• Potential issue with cytoscape & graph tool OSError when running poppunk_visualise with cytoscape even in emulated osx-64 #210
• Check Ns in reads --prop-n flag and ambiguous bases #207
• Check the indiv refine test output, stderr looks wrong
• Merge in pip installable poppunk, almost #209
• Test search range for boundary iterate is fixed & correct
• Check cutoff clusters actually contains some output
• Add documentation for new QC mode and options

[muppi1993]

• Change this line to

  if type(top_value) == str:
      sketch_dict = json.loads(top_value)
  else:
      sketch_dict = top_value
  

[johnlees]

@nickjcroucher Would you be able to confirm the MST -> phylogeny is still working as expected here?

[johnlees]

Example for the merged clusters

Reference database: /media/mirrored-hdd/shorsfield/jobs/poppunk-models/Salmonella/sal_sketch40k
Model fit: /media/mirrored-hdd/shorsfield/jobs/poppunk-models/dbscan_indiv_refine_both
query file: /media/mirrored-hdd/shorsfield/jobs/poppunk-models/Salmonella_query_popPUNK_ids_split.txt.acaaae
command: poppunk_assign --db sal_sketch40k --query Salmonella_query_popPUNK_ids_split.txt.acaaae --model-dir dbscan_indiv_refine_both --core --threads 20 --output query-assignments_acaaae --qc-filter prune

[johnlees]

Just need to have a look at that stall, which may be a sketchlib issue

[johnlees]

🤯

[nickjcroucher]

Fastastic!