Multithreaded file and string consumption

[Adamtaranto]

How fast can we make kmer counting in Rust?

@ctb, are there examples of this from Sourmash or other libraries?

[ctb]

yes, it's actually really easy ;). The trick is to make sure we're using iter and closures wherever possible; then we will just change iter to par_iter, include rayon's prelude, and voila.

This was next on my list after #10 gets finished off.

[Adamtaranto]

Might also save some time if we revcomp the full DNA seq once and pass a sliding window backwards through it (as in sourmash seqtohashes) instead of calculating rc for every kmer.

[Adamtaranto]

@ctb what do you think about making the user specify a thread number for rayon? I think it will try to use all available by default.

[Adamtaranto]

Some other Rust kmer counting projects for ideas:

• Krust
• KmerUtils uses bloom filters
• Kmers