dRAID implementation (WIP)

This patch implements the dRAID vdev driver and a new rebuild
mechanism. This is still work in progress the user interface
may change as well as the on-disk format.

Code changes:
+ module/zfs/vdev_draid.c: vdev driver for dRAID and dRAID spare
+ module/zfs/vdev_scan.c: sequential rebuild for dRAID and mirror
+ cmd/draidcfg/*.[ch]: user space tools, mainly to create permutations
+ module/zfs/vdev_raidz.c: the parity functions need to include dRAID
  skip sectors for computation and reconstruction.
+ module/zfs/vdev_mirror.c: minor changes to support mirror_map_t
  allocated by dRAID vdev (for hybrid mirror support)
+ module/zfs/metaslab.c: to add support for dRAID hybrid mirror, also
  disallow block allocation during rebuild

The following new ztest command options were added to test dRAID.
  -K draid|raidz|random -- kind of RAID to test
  -D <value> -- dRAID data drives per redundancy group
  -G <value> -- dRAID redundancy group count
  -S <value> -- dRAID distributed spare drives
  -R <value> -- RAID parity (raidz or dRAID)
  -L         -- (Existing -G (dump log option) was renamed -L)

Added dRAID specific section to zloop to exercise dRAID across a
wide range of configuration parameters. For example:

  ztest -VVVV -K draid -D 7 -G 6 -S 1 -m 0 -r 1 -R 1 -v 0 -a 12 -s 384m

The goal is to change existing code in a way that when draid is not
in use the effective change is none.

Todo:
* Merge the draidcfg utility into zpool command so that there is no
  additional step before creating a draid pool
* Update the new sequential rebuild code to use the generic mechanism
  for enable/disabling allocations to metaslabs.
* Debug dRAID hybrid mirror code (currently disabled).
* Clean up rebuild in-memory and persisted states so that it doesn't
  share any state with the resilver states. This will allow concurrent
  rebuild and resilver, e.g. a raidz vdev can be resilvering while a
  draid vdev is rebuilding.
* Extend the ZTS to include dRAID test coverage.

Co-authored-by: Isaac Huang <[email protected]>
Co-authored-by: Mark Maybee <[email protected]>
Co-authored-by: Don Brady <[email protected]>
Co-authored-by: Srikanth N S <[email protected]>
Co-authored-by: Stuart Maybee <[email protected]>
Co-authored-by: Brian Behlendorf <[email protected]>
Signed-off-by: Brian Behlendorf <[email protected]>

External-issue: ZFS-12 ZFS-35 ZFS-36 ZFS-17 ZFS-56 ZFS 95 ZFS-96
External-issue: ZFS-100 ZFS-103 ZFS-106 ZFS-110 ZFS-111 ZFS-117
Issue openzfs#9558

cmd/Makefile.am

@@ -1,5 +1,5 @@
 SUBDIRS  = zfs zpool zdb zhack zinject zstreamdump ztest
-SUBDIRS += fsck_zfs vdev_id raidz_test zgenhostid
+SUBDIRS += fsck_zfs vdev_id raidz_test zgenhostid draidcfg
 
 if USING_PYTHON
 SUBDIRS += arcstat arc_summary dbufstat

cmd/draidcfg/.gitignore

@@ -0,0 +1 @@
+/draidcfg

cmd/draidcfg/Makefile.am

@@ -0,0 +1,20 @@
+include $(top_srcdir)/config/Rules.am
+
+AM_CPPFLAGS += -DDEBUG
+
+DEFAULT_INCLUDES += \
+	-I$(top_srcdir)/include \
+	-I$(top_srcdir)/lib/libspl/include
+
+bin_PROGRAMS = draidcfg
+
+draidcfg_SOURCES = \
+	draidcfg.c \
+	draid_permutation.c \
+	draid_permutation.h
+
+draidcfg_LDADD = \
+	$(top_builddir)/lib/libnvpair/libnvpair.la \
+	$(top_builddir)/lib/libzpool/libzpool.la \
+	$(top_builddir)/lib/libzfs/libzfs.la
+draidcfg_LDADD += -lm