ZFS send dedup multi-thread

[mcr-ksh]

Hi,

the "zfs send -vD" is dedup of the stream which is performing very badly. I figured it uses just one core of a multi-core system. So if we have a 28 core machine we wouldn't see any improvement.

Therefore, the feature request to make zfs send dedup multi-threaded.

Best.

[ryao]

Multithreading might not help as much as you think. A known issue when doing zfs send | ... | zfs recv is that the zfs recev part is bursty. zfs recv will read the data stream one transaction at a time, zfs send blocks when a small 64KB receive buffer fills and each transaction is almost always bigger than 64KB. The effect is that the sending and receiving systems will take turns doing work, which is slow.

There is a neat program called mbuffer that allows you to create an additional buffer of a size that you specify. If you were to use it on the zfs recv side of the transfer, it should become much faster.

[wrouesnel]

This is a very naive comment, but couldn't we simply bump the size of that buffer to something larger then most TCP window sizes over say, 10 GbE ethernet, and call it a day as a simple fix?

[akatrevorjay]

Instead of mbuffer an option is to use pv as it is both in existing Linux
repos and supports cool things like a progress bar ;)
On Nov 26, 2012 12:40 PM, "Richard Yao" [email protected] wrote:

Multithreading might not help as much as you think. A known issue when
doing zfs send | ... | zfs recv is that the zfs recev part is bursty. zfs
recv will read the data stream one transaction at a time, zfs send blocks
when a small 64KB receive buffer fills and each transaction is almost
always bigger than 64KB. The effect is that the sending and receiving
systems will take turns doing work, which is slow.

There is a neat program called mbuffer that allows you to create an
additional buffer of a size that you specify. If you were to use it on the zfs
recv side of the transfer, it should become much faster.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/1112#issuecomment-10725348.

[DeHackEd]

'zfs receive' can stall for more than a few seconds while it does a bunch of reads on its local disks, especially for incremental receive jobs. mbuffer, with a default buffer size of over 80 megabytes, makes a lot of sense especially over relatively slow (eg: 100 megabit) networks. TCP buffers alone is not a good idea.

[ryao]

If I recall correctly, mbuffer's default buffer size is 2MB. As for increasing the buffer size, the buffer is in your shell (e.g. bash), so it is not something that we can control unless we add another buffer. That is what mbuffer does.

[ryao]

5c3f61e implemented a larger receive buffer.

[behlendorf]

b738bc5 also significantly improved performance by improving the prefetch.