Implement -t option to zpool create for temporary pool names

[ryao]

Creating virtual machines that have their rootfs on ZFS on hosts that
have their rootfs on ZFS causes SPA namespace collisions when the
standard name rpool is used. The solution is either to give each guest
pool a name unique to the host, which is not always desireable, or boot
a VM environment containing an ISO image to install it, which is
cumbersome.

26b42f3 introduced zpool import -t ... to simplify situations where a host must access a guest's pool when
there is a SPA namespace conflict. We build upon that to introduce
zpool import -t tname .... That allows us to create a pool whose
in-core name is tname, but whose on-disk name is the normal name
specified.

This simplifies the creation of machine images that use a rootfs on ZFS.
That benefits not only real world deployments, but also ZFSOnLinux
development by decreasing the time needed to perform rootfs on ZFS
experiments.

Signed-off-by: Richard Yao [email protected]

[ryao]

To provide an example of where this is helpful, here are some commands from my workstation to setup a VM as the root user:

# Create zvol
zfs create -V 20G -b 4k -o dedup=off rpool/KVM/gzfs

# Create pool
zpool create -R /mnt/gentoo -o cachefile=/tmp/zpool.cache -O normalization=formD -m none -t install rpool rpool/KVM/gzfs

# Create rootfs
zfs create -o mountpoint=none install/ROOT
zfs create -o mountpoint=/ install/ROOT/gentoo

# Create home directories
zfs create -o mountpoint=/home install/HOME
zfs create -o mountpoint=/root install/HOME/root

# Create portage directories
zfs create -o mountpoint=none -o setuid=off install/GENTOO
zfs create -o mountpoint=/usr/portage -o atime=off install/GENTOO/portage
zfs create -o mountpoint=/usr/portage/distfiles install/GENTOO/distfiles

# Create portage build directory
zfs create -o mountpoint=/var/tmp/portage -o compression=lz4 -o sync=disabled install/GENTOO/build-dir

# Create optional packages directory
zfs create -o mountpoint=/usr/portage/packages install/GENTOO/packages

# Create optional ccache directory
zfs create -o mountpoint=/var/tmp/ccache -o compression=lz4 install/GENTOO/ccache

# Store temporary files in /tmp
cd /tmp

# Download stage3
wget 'ftp://gentoo.osuosl.org/pub/gentoo/releases/amd64/autobuilds/current-stage3-amd64/stage3-amd64-[0-9]*.tar.bz2'

# Extract stage3
tar -xvjpf stage3-amd64-*.tar.bz2 -C /mnt/gentoo

# Copy resolv.conf into chroot
cp /etc/resolv.conf /mnt/gentoo/etc/resolv.conf

# Use a mount namespace (replaces chroot method)
unshare -m /bin/bash

# Setup the namespace (this replaces a chroot)
pivot_root /mnt/gentoo /mnt/gentoo/mnt
mount --rbind {/mnt,}/dev                                                                                                                                                                                                                       
mount --rbind {/mnt,}/sys
mount -t proc none /proc
umount -l /mnt
exec bash
PS1="(gentoo-zfs) $PS1"
cd

# Get portage snapshot (use OSUOSL mirror because it is usually fast)
env GENTOO_MIRRORS="http://gentoo.osuosl.org" emerge-webrsync

# Workaround bug #514112
sed -i -e 's/\(epatch_user\)\(.*\)/\1/' /usr/portage/sys-kernel/genkernel/genkernel-3.4.49.2.ebuild
ebuild /usr/portage/sys-kernel/genkernel/genkernel-3.4.49.2.ebuild digest

# Install genkernel
emerge sys-kernel/genkernel

# Install sources
emerge sys-kernel/gentoo-sources

# Build initial kernel (required for checks in sys-kernel/spl and sys-fs/zfs)
# FIXME: Make genkernel support modules_prepare
genkernel kernel --no-clean --no-mountboot

# Install ZFS
echo "sys-kernel/spl ~amd64" >> /etc/portage/package.accept_keywords
echo "sys-fs/zfs-kmod ~amd64" >> /etc/portage/package.accept_keywords
echo "sys-fs/zfs ~amd64" >> /etc/portage/package.accept_keywords
emerge sys-fs/zfs

# Add zfs to boot runlevel
rc-update add zfs boot

# Install GRUB2
echo "sys-boot/grub:2 libzfs" >> /etc/portage/package.use
echo "sys-boot/grub:2 ~amd64" >> /etc/portage/package.accept_keywords
emerge sys-boot/grub:2
touch /etc/mtab
grub2-install /dev/zvol/rpool/KVM/gzfs

# Comment the BOOT, ROOT and SWAP lines in /etc/fstab
sed -i -e "s/\(.*\)\/\(BOOT\|ROOT\|SWAP\)\(.*\)/\#\1\/\2\3/g" /etc/fstab

# Setup serial terminals
sed -i -e 's\#s0:12345:respawn:/sbin/agetty -L 115200 ttyS0 vt100\s0:12345:respawn:/sbin/agetty -a root -L 115200 ttyS0 linux\' /etc/inittab 
sed -i -e 's\#s1:12345:respawn:/sbin/agetty -L 115200 ttyS1 vt100\s1:12345:respawn:/sbin/agetty -a root -L 115200 ttyS1 linux\' /etc/inittab

# Configure GRUB2 to boot the system on serial ports
sed -i -e 's:\(#\|\)\(GRUB_TERMINAL=\)\(.*\)$:\2serial\nGRUB_SERIAL_COMMAND="serial --speed=115200 --unit=0 --word=8 --parity=no --stop=1":' /etc/default/grub
sed -i -e 's:\(#\|\)\(GRUB_CMDLINE_LINUX_DEFAULT=\)\(.*\)$:\2"console=ttyS0,115200n8":' /etc/default/grub

# We must create an empty configuration file so genkernel finds the right one.
touch /boot/grub/grub.cfg

# Setup a hostid:
printf "$(hostid | sed 's/\([0-9A-F]\{2\}\)/\\x\1/gI')" > /etc/hostid

# Build kernel and initramfs
genkernel all --no-clean --no-mountboot --zfs --bootloader=grub2 --callback="emerge @module-rebuild"

# Exit namespace (automatically unmounts /mnt/gentoo/{dev, proc, sys})
exit

# Finalize pool
zfs snapshot install@install
zpool export install
zfs snapshot rpool/KVM/gzfs@install

Then to start the VM as a user:

sudo env KVM_USER=${USER} qtap-manipulate create gzfs
sudo chown ${USER} $(readlink -f /dev/zvol/rpool/KVM/gzfs)
qemu-system-x86_64 -enable-kvm -smp 12,cores=6,threads=2,sockets=1 -m 2048 -device ahci,id=ide -device ide-drive,bus=ide.2,drive=HDD -drive file=/dev/zvol/rpool/KVM/gzfs,id=HDD,if=none -net nic,model=e1000,macaddr=52:54:00:00:ee:40 -net tap,ifname=gzfs,script=no,downscript=no -serial mon:stdio -serial pty -nographic -vga none

This patch eliminates the need to do a machine context switch to a VM environment before the guest is installed. Consequently, I can now quickly create VMs with a rootfs on ZFS to test ideas without having to utilize an emulated graphical console. The only caveat is that the VMs created this way do not have whole_disk set or a partition table created, but that is a relatively minor detail when I care about debugging the kernel code during early boot. Having to juggle guest pool names is far more annoying.

[sempervictus]

Seems to work like a charm, thank you.
One thing i did notice though is that zdb gets a bit confused when called with no arguments. It believes rpool to be the newly created pool with that on-disk name. Could this cause potential issues elsewhere in ZFS if it wants to do something with rpool?

[ryao]

This merits some comment. While temporary names are useful for both creation and import, the API for the import case is easier to use than the API for the create case. On import, the API already supported two names and import flags. Consequently, -t could just set a bit in the import flags to state that the old name is to be sent to disk. On create, the API only uses 1 name and opts for a nvlist does not support "creation flags", such that we are forced to use the nvlist. We have validation code in userland and a lingering question on my mind is if the import code should handle passing this flag via a nvlist when doing pool import. If it is, then some additional code should be added to spa_import() to handle this. If not, then this userland validation code should to change.

[ryao]

@sempervictus Thanks for catching that. This should also affect zpool import -t, but this was easy to miss in testing because -t is intended to be used with -R. I have pushed a fix to this pull request that fixes -t to make ZPOOL_PROP_CACHEFILE default to none when -t is used as is the case with -R. I have also fixed the proposed zpool create -t patch to do the same.

[sempervictus]

Merging and testing. Thank you, as always.
I actually ate a bunch of that mess soon as i started abusing the new functionality to seed my rpool to several other bootable things. Ended up having to destroy the cachefile in chroot environments or face the conflict while still using initrd and its limited tools. Still, creating rpools and sending my root dataset there has been very useful.

[behlendorf]

@ryao when you get a minute could you refresh this against master. @sempervictus how has this been working for you?

[ryao]

@behlendorf It is refreshed.

[behlendorf]

@ryao Thanks, according to a quick glance at the buildbot results, 65518ae, has introduced a memory leak which will need to be run down.

[ryao]

@behlendorf The problem is that spa_add() will make a copy of the nvlist_t passed to it such that the caller is responsible for freeing it. I had wrongly assumed that such an allocation would not occur. Consequently, adding a free should be sufficient to correct the leak that this introduced.

[ryao]

@behlendorf The failures appear to be pre-existing issues in the build bot. This should be safe to merge.

[ryao]

@behlendorf I made a slight change in the placement of code with respect to a comment in module/zfs/spa.c when using this as a test case for the process of sending changes back to Illumos. I have updated the pull request. Hopefully, pre-existing issues affecting the buildbot will not stop it from being marked green this time.

[behlendorf]

This looks good to me. As noted the buildbot failures were unrelated.

[ryao]

That is good to hear. :)

[behlendorf]

Then you're going to love this. Merged as:

83e9986 Implement -t option to zpool create for temporary pool names
00d2a8c zpool import -t should not update cachefile
2f3ec90 Add add_prop_list_default helper

[ryao]

@behlendorf That made my day. :)