zpool remove of top-level mirror or vdev still no work #13552
Comments
|
Well, you're running ZFS 2.0.6 kernel modules with ZFS 2.1.4 userland, so surprising things may ensue. Did you install jonathonf's PPA but not install the zfs-dkms package with the others, maybe? That's usually what happened when I see that. It would be helpful if you included the (full, not just the parts you think are relevant) output of Finally, an -f to override the space checks would be a very bad idea, IMO - people wouldn't bother reporting issues or investigating why it refused, they'd just start using -f, and then complain when it broke midway through if they did, indeed, not have sufficient space. ( e: You also don't inherently have to lose ~any space going from "full disk" to "partition", AFAIK - since "full disk" just means "we made the partition table, set a bit on the device metadata that we manage the partition table and can rewrite it, and hide the 'part1' or similar in the |
|
Retried with fresh install same FAIL outcome — it had also failed with two sets of mirrors prior |
|
That's the same kind of mismatched kernel module version as before, and 22.04 doesn't ship that kernel version, so I think you installed something else. |
|
Fixed that, too: unfortunately still there. I will try the second pool, too |
|
The purpose of this exercise was to change the vdevs from being whole disk to instead the single partition on that same disk. Unexpectedly, I could add a smaller device to the first stripe making it a mirror. It appears zfs is confused about vdev sizes The goal below is for zro to consist of the single vdev 202d6b64 I do have off-line backup, so once layout is satisfactory, the data can be verified |
|
Now this command fails: when it should work. Below is vdev sizes: |
|
Hey, I managed to fix it!! I added a random 426 GiB ssd stripe to try to remove the big bulky duplicated vdev After that, both stripes could be removed, leaving me with the pool organization I wanted Free space on the final pool is 153 GiB of 1.81 TiB |
|
After completion, I could verify the data to be OK using: |
|
Thank you @haraldrudell for your tip on adding MORE space to then REMOVE space :-D This worked for me, too. I will detail my experience below for @behlendorf and @ahrens to consider: Context, all my HDDs are in USB boxes outside - this is sub ideal, but it's the best I have for now. My full set is as follows: 2.0.4 from the PPA on ubu 20.04 - this is what I installed on and I don't want to change without some off-site snapshot sync going on first. 3x zpools:
I had to shut down the whole machine for a few weeks and brought it up again last night for the first time and had trouble with 3 out of 4 of the USB HDDs. The two BIG ones had to resilver from the big SSD once I told them to come back online, but it didn't take super long since they're fairly empty. Not ideal, and a function of not doing an offline operation prior to shut down and an improper unmount AFAICT. Work in progress. One of the 4TB HDDs was fine in the 2-way mirror and allowed that zpool and the FSes within to function in a degraded state. The other 4TB HDD was suddenly called sde rather than the ZFS ID or the disk ID which I set them up with. This one was unhappy and FAULTED. I tried power cycling the drive and other stuff, but no dice. Remediation process: I incorrectly thought
Which left me where I am with 2 copies of my data and a third recreating on a brand new ironwolf pro 6TB and and the old "bad" 4TB available to do what I please with: add to the SSD to create a hybrid mirror so my snapshot etiquette isn't as critical, similar to the big 3 way mirror. Observations:
I'm happy now but I would describe this as a really poor user experience triggered by my own shoddy USB set up and my own misuse of commands, however recovering was way harder than it needed to be because of the combo of Thanks @haraldrudell for documenting your experience here, I thought your fix was crazy, but it seemed to make sense, and in the end it worked perfectly for me. Thanks @behlendorf and @ahrens for all your hard work that allowed me to have this stack in the first place. Much appreciated. Hopefully this monologue is in some way useful. Cheers. |
System information
Describe the problem you're observing
zpool remove errors despite documented requirements apparently fulfilled
Describe how to reproduce the problem
use my pool…
zfs provides no ability for more detail, any log or any helpful error messages
For example, zfs could tell me how many more bytes I need, right?
Include any warning/errors/backtraces from the system logs
I am trying to migrate a pool from disk to partition
To remove a top-level mirror or simple vdev (I tried both) ashift must be same, no raidz, encryption keys loaded
THE ERROR
THE FACTS
NOTE: 202d… is apparently larger than b7f9… Data used to be stored on bf79… alone, and now it does not fit onto a larger vdev?
HERE’S THE SIZE OF 202d… as a POOL:
APPARENT SPACE AVAILABLE:
1992864825344 - 1828179083264 = 164685742080 bytes ≈ 153 GiB
PERCENTAGE SPACE AVAILABLE:
164685742080 / 1992864825344 ≈ 8.2%
#11409 lists some apparently undocumented requirements
#11356 have some more
Because the second device is actually larger, this is probably some other buggy bug hiding behind the out of space error
I noticed you lose about 16 GiB by going to partition from full disk
zdb -C also lists 3 indirect devices that should be removable using zfs remap That command is not available
SUGGESTION
for zpool remove to have a -f flag that ignores these highly inaccurate predictions: if it fails, it fails. remove is already aborted if an i/o error is encountered. All it costs to try is to wait for it to fail
make error message actionable: how many bytes, what are the determined values
The text was updated successfully, but these errors were encountered: