Illumos 4976-4984 - metaslab improvements

[dweeezil]

4976 zfs should only avoid writing to a failing non-redundant top-level vdev
4978 ztest fails in get_metaslab_refcount()
4979 extend free space histogram to device and pool
4980 metaslabs should have a fragmentation metric
4981 remove fragmented ops vector from block allocator
4982 space_map object should proactively upgrade when feature is enabled
4983 need to collect metaslab information via mdb
4984 device selection should use fragmentation metric
Reviewed by: Matthew Ahrens [email protected]
Reviewed by: Adam Leventhal [email protected]
Reviewed by: Christopher Siden [email protected]
Approved by: Garrett D'Amore [email protected]

References:
https://www.illumos.org/issues/4976
https://www.illumos.org/issues/4978
https://www.illumos.org/issues/4979
https://www.illumos.org/issues/4980
https://www.illumos.org/issues/4981
https://www.illumos.org/issues/4982
https://www.illumos.org/issues/4983
https://www.illumos.org/issues/4984

Notes:
The "zdb -M" option has been re-tasked to display the new metaslab
fragmentation metric and the new "zdb -I" option is used to control
the maximum number of in-flight I/Os.

The new fragmentation metric is derived from the space map histogram
which has been rolled up to the vdev and pool level and is presented
to the user via "zpool list".

Add a number of module parameters related to the new metaslab weighting
logic.

Ported by: Tim Chase [email protected]

[dweeezil]

There may be a few more module parameters to add but I've added the most important ones. I'm posting this pull request so it can hopefully get a bit of testing by anyone interested in making the spacemap histograms do something more interesting.

The spacemap histograms should be read-compatible and allow grub booting but I've not tried it yet.

@behlendorf I had to wrap the vdev_get_stats() calls in the sync task with spa_config_enter() and spa_config_exit() in order to avoid an assert. I'm not sure how bad of an idea this is nor how much overhead it might incur but I figured I'd point it out since you added these calls to provide the stats visibility.

[dweeezil]

I added dweeezil/zfs@3a5fa41 after encountering the assert in vdev_dirty() during manual testing of upgrading an existing pool. The metaslab upgrade can't be performed in SPA_LOAD_TRYIMPORT mode. It would seem this fix should also be in the upstream code.

[behlendorf]

@dweeezil I want to do a little more manual testing but thus far everything looks good. Just a the few minor review comments above.

[dweeezil]

@behlendorf I presume you'll take care of these issues before merging. I'm going to exercise it a bit more over the weekend. I think this is the last of the major illumos patches we're missing.

[behlendorf]

Yeah. I'll make those few changes when this is merged. Most likely on Monday.

We may also want to take a look at the multi vdev crash feature just for the sake of compatibility.

[behlendorf]

Merged as:

8b0a084 Don't upgrade a metaslab when the pool is not writable
f3a7f66 Illumos 4976-4984 - metaslab improvements