Ignore special vdev ashift for spa ashift min/max

[don-brady]

Motivation and Context

The removal of a vdev in the normal class would fail if there was a special or deup vdev that had a different ashift than the vdevs in the normal class.

Addresses issue #9363 and issue #9364

Description

Moved the initialization of spa_min_ashift / spa_max_ashift from vdev_open so that it occurs after the vdev allocation bias was initialized (i.e. after vdev_load).

Caveat -- In order to remove a special/dedup vdev it must have the same ashift as the normal pool vdevs.
This could perhaps be lifted in the future (i.e. for the case where there is ample space in any surviving special class vdevs).

How Has This Been Tested?

ZTS -- ran functional tests for alloc_class and removal.
ztest runs
Manual run of failure case:

$ sudo zpool create vote sdb sdc sdd
$ sudo zdb -l /dev/sdb1 | grep ashift
        ashift: 9
$ sudo zpool add -o ashift=12 vote special sdf
$ sudo zdb -l /dev/sdf1 | grep ashift
        ashift: 12
$ sudo zpool export vote
$ sudo zpool import vote
$ sudo zpool status -P vote
  pool: vote
 state: ONLINE
config:

	NAME         STATE     READ WRITE CKSUM
	vote         ONLINE       0     0     0
	  /dev/sdb1  ONLINE       0     0     0
	  /dev/sdc1  ONLINE       0     0     0
	  /dev/sdd1  ONLINE       0     0     0
	special
	  /dev/sdf1  ONLINE       0     0     0

errors: No known data errors
$ sudo zpool remove vote sdd

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Performance enhancement (non-breaking change which improves efficiency)
• Code cleanup (non-breaking change which makes code smaller or more readable)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation (a change to man pages or other documentation)

Checklist:

• My code follows the OpenZFS code style requirements.
• I have updated the documentation accordingly.
• I have read the contributing document.
• I have added tests to cover my changes.
• I have run the ZFS Test Suite with this change applied.
• All commit messages are properly formatted and contain Signed-off-by.

[codecov]

Codecov Report

Merging #11053 into master will increase coverage by 0.15%.
The diff coverage is 80.00%.

@@            Coverage Diff             @@
##           master   #11053      +/-   ##
==========================================
+ Coverage   79.66%   79.82%   +0.15%     
==========================================
  Files         397      397              
  Lines      125753   125755       +2     
==========================================
+ Hits       100184   100383     +199     
+ Misses      25569    25372     -197     

Flag	Coverage Δ
#kernel	80.39% <80.00%> (+<0.01%)	⬆️
#user	66.02% <50.00%> (+0.39%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
module/zfs/vdev.c	89.93% <ø> (+0.15%)	⬆️
module/zfs/vdev_removal.c	96.83% <80.00%> (-0.09%)	⬇️
cmd/zdb/zdb_il.c	30.86% <0.00%> (-24.08%)	⬇️
module/lua/lmem.c	83.33% <0.00%> (-4.17%)	⬇️
module/zfs/vdev_rebuild.c	93.69% <0.00%> (-3.48%)	⬇️
module/zfs/spa_log_spacemap.c	93.40% <0.00%> (-2.30%)	⬇️
lib/libzutil/os/linux/zutil_import_os.c	80.06% <0.00%> (-0.67%)	⬇️
cmd/zed/agents/zfs_mod.c	76.89% <0.00%> (-0.67%)	⬇️
module/zfs/vdev_raidz.c	91.31% <0.00%> (-0.66%)	⬇️
module/zfs/vdev_initialize.c	97.15% <0.00%> (-0.64%)	⬇️
... and 54 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 61868bb...b24b3a7. Read the comment docs.