Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ZFS+acltype+fuse-overlayfs, Operation Not Supported #11213

Closed
andrewgdunn opened this issue Aug 12, 2021 · 14 comments
Closed

ZFS+acltype+fuse-overlayfs, Operation Not Supported #11213

andrewgdunn opened this issue Aug 12, 2021 · 14 comments
Labels
locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.

Comments

@andrewgdunn
Copy link

I've been happily cruising along using Fedora+ZFS+Podman to run a variety of services. I am deploying these in what I think is the recommended pattern based on interacting with podman for a little while now. Some other contexts:

  • ZFS installed from the OpenZFS Packaging
  • users are created, then subuid/subgid are added, then linger is enabled
  • users home directories are on ZFS (maybe this is biting me?)
  • create pods/containers by using podman directly, then generate systemd units from those instances

This appears to have manifested after a recent update, all container deployments are impacted on this system.

[root@vault ~]# dnf history info 43
Transaction ID : 43
Begin time     : Wed 11 Aug 2021 04:08:24 PM EDT
Begin rpmdb    : 1020:9b66e7e51456ba22c77419720308627965f412d3
End time       : Wed 11 Aug 2021 04:14:16 PM EDT (5 minutes)
End rpmdb      : 1019:cef866baafb830bae0a92e223378beb4b474a6ff
User           : System <unset>
Return-Code    : Success
Releasever     : 34
Command Line   : upgrade --refresh
Comment        : 
Packages Altered:
    Install       kernel-5.13.8-200.fc34.x86_64                              @updates
    Install       kernel-core-5.13.8-200.fc34.x86_64                         @updates
    Install       kernel-devel-5.13.8-200.fc34.x86_64                        @updates
    Install       kernel-modules-5.13.8-200.fc34.x86_64                      @updates
    Install       libzfs5-2.1.0-1.fc34.x86_64                                @zfs
    Obsoleted     libzfs4-2.0.5-1.fc34.x86_64                                @@System
    Install       libzpool5-2.1.0-1.fc34.x86_64                              @zfs
    Obsoleted     libzpool4-2.0.5-1.fc34.x86_64                              @@System
    Upgrade       at-spi2-core-2.40.3-1.fc34.x86_64                          @updates
    Upgraded      at-spi2-core-2.40.1-1.fc34.x86_64                          @@System
    Upgrade       avahi-libs-0.8-14.fc34.x86_64                              @updates
    Upgraded      avahi-libs-0.8-9.fc34.x86_64                               @@System
    Upgrade       containernetworking-plugins-1.0.0-0.3.rc1.fc34.x86_64      @updates
    Upgraded      containernetworking-plugins-1.0.0-0.2.rc1.fc34.x86_64      @@System
    Upgrade       dnsmasq-2.85-3.fc34.x86_64                                 @updates
    Upgraded      dnsmasq-2.85-1.fc34.x86_64                                 @@System
    Upgrade       fuse-overlayfs-1.7.0-1.fc34.x86_64                         @updates
    Upgraded      fuse-overlayfs-1.5.0-1.fc34.x86_64                         @@System
    Upgrade       glib2-2.68.3-1.fc34.x86_64                                 @updates
    Upgraded      glib2-2.68.2-1.fc34.x86_64                                 @@System
    Upgrade       hwdata-0.350-1.fc34.noarch                                 @updates
    Upgraded      hwdata-0.349-1.fc34.noarch                                 @@System
    Upgrade       libdav1d-0.9.1-1.fc34.x86_64                               @updates
    Upgraded      libdav1d-0.9.0-1.fc34.x86_64                               @@System
    Upgrade       librsvg2-2.50.7-1.fc34.x86_64                              @updates
    Upgraded      librsvg2-2.50.6-1.fc34.x86_64                              @@System
    Upgrade       libxcrypt-4.4.24-1.fc34.x86_64                             @updates
    Upgraded      libxcrypt-4.4.23-1.fc34.x86_64                             @@System
    Upgrade       libxcrypt-compat-4.4.24-1.fc34.x86_64                      @updates
    Upgraded      libxcrypt-compat-4.4.23-1.fc34.x86_64                      @@System
    Upgrade       libxcrypt-devel-4.4.24-1.fc34.x86_64                       @updates
    Upgraded      libxcrypt-devel-4.4.23-1.fc34.x86_64                       @@System
    Upgrade       mesa-filesystem-21.1.6-1.fc34.x86_64                       @updates
    Upgraded      mesa-filesystem-21.1.5-1.fc34.x86_64                       @@System
    Upgrade       mesa-libEGL-21.1.6-1.fc34.x86_64                           @updates
    Upgraded      mesa-libEGL-21.1.5-1.fc34.x86_64                           @@System
    Upgrade       mesa-libGL-21.1.6-1.fc34.x86_64                            @updates
    Upgraded      mesa-libGL-21.1.5-1.fc34.x86_64                            @@System
    Upgrade       mesa-libgbm-21.1.6-1.fc34.x86_64                           @updates
    Upgraded      mesa-libgbm-21.1.5-1.fc34.x86_64                           @@System
    Upgrade       mesa-libglapi-21.1.6-1.fc34.x86_64                         @updates
    Upgraded      mesa-libglapi-21.1.5-1.fc34.x86_64                         @@System
    Upgrade       mesa-vulkan-drivers-21.1.6-1.fc34.x86_64                   @updates
    Upgraded      mesa-vulkan-drivers-21.1.5-1.fc34.x86_64                   @@System
    Upgrade       perl-Archive-Tar-2.40-1.fc34.noarch                        @updates
    Upgraded      perl-Archive-Tar-2.38-4.fc34.noarch                        @@System
    Upgrade       perl-HTTP-Tiny-0.078-1.fc34.noarch                         @updates
    Upgraded      perl-HTTP-Tiny-0.076-458.fc34.noarch                       @@System
    Upgrade       perl-Module-CoreList-1:5.20210723-1.fc34.noarch            @updates
    Upgraded      perl-Module-CoreList-1:5.20210620-1.fc34.noarch            @@System
    Upgrade       perl-Module-CoreList-tools-1:5.20210723-1.fc34.noarch      @updates
    Upgraded      perl-Module-CoreList-tools-1:5.20210620-1.fc34.noarch      @@System
    Upgrade       perl-experimental-0.025-1.fc34.noarch                      @updates
    Upgraded      perl-experimental-0.024-1.fc34.noarch                      @@System
    Upgrade       podman-3:3.2.3-2.fc34.x86_64                               @updates
    Upgraded      podman-3:3.2.3-1.fc34.x86_64                               @@System
    Upgrade       podman-plugins-3:3.2.3-2.fc34.x86_64                       @updates
    Upgraded      podman-plugins-3:3.2.3-1.fc34.x86_64                       @@System
    Upgrade       rav1e-libs-0.4.1-4.fc34.x86_64                             @updates
    Upgraded      rav1e-libs-0.4.1-2.fc34.x86_64                             @@System
    Upgrade       systemd-248.7-1.fc34.x86_64                                @updates
    Upgraded      systemd-248.6-1.fc34.x86_64                                @@System
    Upgrade       systemd-libs-248.7-1.fc34.x86_64                           @updates
    Upgraded      systemd-libs-248.6-1.fc34.x86_64                           @@System
    Upgrade       systemd-networkd-248.7-1.fc34.x86_64                       @updates
    Upgraded      systemd-networkd-248.6-1.fc34.x86_64                       @@System
    Upgrade       systemd-oomd-defaults-248.7-1.fc34.x86_64                  @updates
    Upgraded      systemd-oomd-defaults-248.6-1.fc34.x86_64                  @@System
    Upgrade       systemd-pam-248.7-1.fc34.x86_64                            @updates
    Upgraded      systemd-pam-248.6-1.fc34.x86_64                            @@System
    Upgrade       systemd-rpm-macros-248.7-1.fc34.noarch                     @updates
    Upgraded      systemd-rpm-macros-248.6-1.fc34.noarch                     @@System
    Upgrade       systemd-udev-248.7-1.fc34.x86_64                           @updates
    Upgraded      systemd-udev-248.6-1.fc34.x86_64                           @@System
    Upgrade       vim-common-2:8.2.3290-1.fc34.x86_64                        @updates
    Upgraded      vim-common-2:8.2.3223-1.fc34.x86_64                        @@System
    Upgrade       vim-enhanced-2:8.2.3290-1.fc34.x86_64                      @updates
    Upgraded      vim-enhanced-2:8.2.3223-1.fc34.x86_64                      @@System
    Upgrade       vim-filesystem-2:8.2.3290-1.fc34.noarch                    @updates
    Upgraded      vim-filesystem-2:8.2.3223-1.fc34.noarch                    @@System
    Upgrade       vim-minimal-2:8.2.3290-1.fc34.x86_64                       @updates
    Upgraded      vim-minimal-2:8.2.3223-1.fc34.x86_64                       @@System
    Upgrade       ffmpeg-libs-4.4-6.fc34.x86_64                              @rpmfusion-free-updates
    Upgraded      ffmpeg-libs-4.4-4.fc34.x86_64                              @@System
    Upgrade       libnvpair3-2.1.0-1.fc34.x86_64                             @zfs
    Upgraded      libnvpair3-2.0.5-1.fc34.x86_64                             @@System
    Upgrade       libuutil3-2.1.0-1.fc34.x86_64                              @zfs
    Upgraded      libuutil3-2.0.5-1.fc34.x86_64                              @@System
    Upgrade       zfs-2.1.0-1.fc34.x86_64                                    @zfs
    Upgraded      zfs-2.0.5-1.fc34.x86_64                                    @@System
    Upgrade       zfs-dkms-2.1.0-1.fc34.noarch                               @zfs
    Upgraded      zfs-dkms-2.0.5-1.fc34.noarch                               @@System
    Reason Change Judy-1.0.5-25.fc34.x86_64                                  @fedora
    Removed       kernel-5.12.15-300.fc34.x86_64                             @@System
    Removed       kernel-core-5.12.15-300.fc34.x86_64                        @@System
    Removed       kernel-devel-5.12.15-300.fc34.x86_64                       @@System
    Removed       kernel-modules-5.12.15-300.fc34.x86_64                     @@System
    Removed       kmod-nvidia-5.12.15-300.fc34.x86_64-3:465.31-1.fc34.x86_64 @@System
Scriptlet output:
   1 Loading new zfs-2.1.0 DKMS files...
   2 Building for 5.13.4-200.fc34.x86_64
   3 Building initial module for 5.13.4-200.fc34.x86_64
   4 Done.
   5 
   6 zavl.ko.xz:
   7 Running module version sanity check.
   8  - Original module
   9    - This kernel never originally had a module by this name
  10  - Installation
  11    - Installing to /lib/modules/5.13.4-200.fc34.x86_64/extra/
  12 
  13 znvpair.ko.xz:
  14 Running module version sanity check.
  15  - Original module
  16    - This kernel never originally had a module by this name
  17  - Installation
  18    - Installing to /lib/modules/5.13.4-200.fc34.x86_64/extra/
  19 
  20 zunicode.ko.xz:
  21 Running module version sanity check.
  22  - Original module
  23    - This kernel never originally had a module by this name
  24  - Installation
  25    - Installing to /lib/modules/5.13.4-200.fc34.x86_64/extra/
  26 
  27 zcommon.ko.xz:
  28 Running module version sanity check.
  29  - Original module
  30    - This kernel never originally had a module by this name
  31  - Installation
  32    - Installing to /lib/modules/5.13.4-200.fc34.x86_64/extra/
  33 
  34 zfs.ko.xz:
  35 Running module version sanity check.
  36  - Original module
  37    - This kernel never originally had a module by this name
  38  - Installation
  39    - Installing to /lib/modules/5.13.4-200.fc34.x86_64/extra/
  40 
  41 icp.ko.xz:
  42 Running module version sanity check.
  43  - Original module
  44    - This kernel never originally had a module by this name
  45  - Installation
  46    - Installing to /lib/modules/5.13.4-200.fc34.x86_64/extra/
  47 
  48 zlua.ko.xz:
  49 Running module version sanity check.
  50  - Original module
  51    - This kernel never originally had a module by this name
  52  - Installation
  53    - Installing to /lib/modules/5.13.4-200.fc34.x86_64/extra/
  54 
  55 spl.ko.xz:
  56 Running module version sanity check.
  57  - Original module
  58    - This kernel never originally had a module by this name
  59  - Installation
  60    - Installing to /lib/modules/5.13.4-200.fc34.x86_64/extra/
  61 
  62 zzstd.ko.xz:
  63 Running module version sanity check.
  64 Error! Module version 1.4.5 for zzstd.ko.xz
  65 is not newer than what is already found in kernel 5.13.4-200.fc34.x86_64 (1.4.5).
  66 You may override by specifying --force.
  67 
  68 depmod......
  69 Warning: The unit file, source configuration file or drop-ins of systemd-modules-load.service changed on disk. Run 'systemctl daemon-reload' to reload units.
  70 
  71 DKMS: install completed.
  72 
  73 Uninstall of zfs module (zfs-2.0.5-1) beginning:
  74 
  75 -------- Uninstall Beginning --------
  76 Module:  zfs
  77 Version: 2.0.5
  78 Kernel:  5.12.15-300.fc34.x86_64 (x86_64)
  79 -------------------------------------
  80 
  81 Status: Before uninstall, this module version was ACTIVE on this kernel.
  82 
  83 zavl.ko.xz:
  84  - Uninstallation
  85    - Deleting from: /lib/modules/5.12.15-300.fc34.x86_64/extra/
  86  - Original module
  87    - No original module was found for this module on this kernel.
  88    - Use the dkms install command to reinstall any previous module version.
  89 
  90 
  91 znvpair.ko.xz:
  92  - Uninstallation
  93    - Deleting from: /lib/modules/5.12.15-300.fc34.x86_64/extra/
  94  - Original module
  95    - No original module was found for this module on this kernel.
  96    - Use the dkms install command to reinstall any previous module version.
  97 
  98 
  99 zunicode.ko.xz:
 100  - Uninstallation
 101    - Deleting from: /lib/modules/5.12.15-300.fc34.x86_64/extra/
 102  - Original module
 103    - No original module was found for this module on this kernel.
 104    - Use the dkms install command to reinstall any previous module version.
 105 
 106 
 107 zcommon.ko.xz:
 108  - Uninstallation
 109    - Deleting from: /lib/modules/5.12.15-300.fc34.x86_64/extra/
 110  - Original module
 111    - No original module was found for this module on this kernel.
 112    - Use the dkms install command to reinstall any previous module version.
 113 
 114 
 115 zfs.ko.xz:
 116  - Uninstallation
 117    - Deleting from: /lib/modules/5.12.15-300.fc34.x86_64/extra/
 118  - Original module
 119    - No original module was found for this module on this kernel.
 120    - Use the dkms install command to reinstall any previous module version.
 121 
 122 
 123 icp.ko.xz:
 124  - Uninstallation
 125    - Deleting from: /lib/modules/5.12.15-300.fc34.x86_64/extra/
 126  - Original module
 127    - No original module was found for this module on this kernel.
 128    - Use the dkms install command to reinstall any previous module version.
 129 
 130 
 131 zlua.ko.xz:
 132  - Uninstallation
 133    - Deleting from: /lib/modules/5.12.15-300.fc34.x86_64/extra/
 134  - Original module
 135    - No original module was found for this module on this kernel.
 136    - Use the dkms install command to reinstall any previous module version.
 137 
 138 
 139 spl.ko.xz:
 140  - Uninstallation
 141    - Deleting from: /lib/modules/5.12.15-300.fc34.x86_64/extra/
 142  - Original module
 143    - No original module was found for this module on this kernel.
 144    - Use the dkms install command to reinstall any previous module version.
 145 
 146 
 147 zzstd.ko.xz:
 148  - Uninstallation
 149    - Deleting from: /lib/modules/5.12.15-300.fc34.x86_64/extra/
 150  - Original module
 151    - No original module was found for this module on this kernel.
 152    - Use the dkms install command to reinstall any previous module version.
 153 
 154 depmod......
 155 
 156 DKMS: uninstall completed.
 157 
 158 -------- Uninstall Beginning --------
 159 Module:  zfs
 160 Version: 2.0.5
 161 Kernel:  5.13.4-200.fc34.x86_64 (x86_64)
 162 -------------------------------------
 163 
 164 Status: This module version was INACTIVE for this kernel.
 165 depmod......
 166 
 167 DKMS: uninstall completed.
 168 
 169 -------- Uninstall Beginning --------
 170 Module:  zfs
 171 Version: 2.0.5
 172 Kernel:  5.13.6-200.fc34.x86_64 (x86_64)
 173 -------------------------------------
 174 
 175 Status: Before uninstall, this module version was ACTIVE on this kernel.
 176 
 177 zavl.ko.xz:
 178  - Uninstallation
 179    - Deleting from: /lib/modules/5.13.6-200.fc34.x86_64/extra/
 180  - Original module
 181    - No original module was found for this module on this kernel.
 182    - Use the dkms install command to reinstall any previous module version.
 183 
 184 
 185 znvpair.ko.xz:
 186  - Uninstallation
 187    - Deleting from: /lib/modules/5.13.6-200.fc34.x86_64/extra/
 188  - Original module
 189    - No original module was found for this module on this kernel.
 190    - Use the dkms install command to reinstall any previous module version.
 191 
 192 
 193 zunicode.ko.xz:
 194  - Uninstallation
 195    - Deleting from: /lib/modules/5.13.6-200.fc34.x86_64/extra/
 196  - Original module
 197    - No original module was found for this module on this kernel.
 198    - Use the dkms install command to reinstall any previous module version.
 199 
 200 
 201 zcommon.ko.xz:
 202  - Uninstallation
 203    - Deleting from: /lib/modules/5.13.6-200.fc34.x86_64/extra/
 204  - Original module
 205    - No original module was found for this module on this kernel.
 206    - Use the dkms install command to reinstall any previous module version.
 207 
 208 
 209 zfs.ko.xz:
 210  - Uninstallation
 211    - Deleting from: /lib/modules/5.13.6-200.fc34.x86_64/extra/
 212  - Original module
 213    - No original module was found for this module on this kernel.
 214    - Use the dkms install command to reinstall any previous module version.
 215 
 216 
 217 icp.ko.xz:
 218  - Uninstallation
 219    - Deleting from: /lib/modules/5.13.6-200.fc34.x86_64/extra/
 220  - Original module
 221    - No original module was found for this module on this kernel.
 222    - Use the dkms install command to reinstall any previous module version.
 223 
 224 
 225 zlua.ko.xz:
 226  - Uninstallation
 227    - Deleting from: /lib/modules/5.13.6-200.fc34.x86_64/extra/
 228  - Original module
 229    - No original module was found for this module on this kernel.
 230    - Use the dkms install command to reinstall any previous module version.
 231 
 232 
 233 spl.ko.xz:
 234  - Uninstallation
 235    - Deleting from: /lib/modules/5.13.6-200.fc34.x86_64/extra/
 236  - Original module
 237    - No original module was found for this module on this kernel.
 238    - Use the dkms install command to reinstall any previous module version.
 239 
 240 
 241 zzstd.ko.xz:
 242  - Uninstallation
 243    - Deleting from: /lib/modules/5.13.6-200.fc34.x86_64/extra/
 244  - Original module
 245    - No original module was found for this module on this kernel.
 246    - Use the dkms install command to reinstall any previous module version.
 247 
 248 depmod......
 249 
 250 DKMS: uninstall completed.
 251 
 252 ------------------------------
 253 Deleting module version: 2.0.5
 254 completely from the DKMS tree.
 255 ------------------------------
 256 Done.
 257 warning: file /lib/modules/5.12.15-300.fc34.x86_64/updates: remove failed: No such file or directory
 258 dkms: running auto installation service for kernel 5.13.8-200.fc34.x86_64
 259 
 260 Kernel preparation unnecessary for this kernel.  Skipping...
 261 
 262 Running the pre_build script:
 263 checking for gawk... gawk
 264 checking metadata... META file
 265 checking build system type... x86_64-pc-linux-gnu
 266 checking host system type... x86_64-pc-linux-gnu
 267 checking target system type... x86_64-pc-linux-gnu
 268 checking whether to enable maintainer-specific portions of Makefiles... no
 269 checking whether make supports nested variables... yes
 270 checking for a BSD-compatible install... /bin/install -c
 271 checking whether build environment is sane... yes
 272 checking for a thread-safe mkdir -p... /bin/mkdir -p
 273 checking whether make sets $(MAKE)... yes
 274 checking how to print strings... printf
 275 checking whether make supports the include directive... yes (GNU style)
 276 checking for gcc... gcc
 277 checking whether the C compiler works... yes
 278 checking for C compiler default output file name... a.out
 279 checking for suffix of executables... 
 280 checking whether we are cross compiling... no
 281 checking for suffix of object files... o
 282 checking whether we are using the GNU C compiler... yes
 283 checking whether gcc accepts -g... yes
 284 checking for gcc option to accept ISO C89... none needed
 285 checking whether gcc understands -c and -o together... yes
 286 checking dependency style of gcc... gcc3
 287 checking for a sed that does not truncate output... /bin/sed
 288 checking for grep that handles long lines and -e... /bin/grep
 289 checking for egrep... /bin/grep -E
 290 checking for fgrep... /bin/grep -F
 291 checking for ld used by gcc... /bin/ld
 292 checking if the linker (/bin/ld) is GNU ld... yes
 293 checking for BSD- or MS-compatible name lister (nm)... /bin/nm -B
 294 checking the name lister (/bin/nm -B) interface... BSD nm
 295 checking whether ln -s works... yes
 296 checking the maximum length of command line arguments... 1572864
 297 checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop
 298 checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop
 299 checking for /bin/ld option to reload object files... -r
 300 checking for objdump... objdump
 301 checking how to recognize dependent libraries... pass_all
 302 checking for dlltool... no
 303 checking how to associate runtime and link libraries... printf %s\n
 304 checking for ar... ar
 305 checking for archiver @FILE support... @
 306 checking for strip... strip
 307 checking for ranlib... ranlib
 308 checking command to parse /bin/nm -B output from gcc object... ok
 309 checking for sysroot... no
 310 checking for a working dd... /bin/dd
 311 checking how to truncate binary pipes... /bin/dd bs=4096 count=1
 312 checking for mt... no
 313 checking if : is a manifest tool... no
 314 checking how to run the C preprocessor... gcc -E
 315 checking for ANSI C header files... yes
 316 checking for sys/types.h... yes
 317 checking for sys/stat.h... yes
 318 checking for stdlib.h... yes
 319 checking for string.h... yes
 320 checking for memory.h... yes
 321 checking for strings.h... yes
 322 checking for inttypes.h... yes
 323 checking for stdint.h... yes
 324 checking for unistd.h... yes
 325 checking for dlfcn.h... yes
 326 checking for objdir... .libs
 327 checking if gcc supports -fno-rtti -fno-exceptions... no
 328 checking for gcc option to produce PIC... -fPIC -DPIC
 329 checking if gcc PIC flag -fPIC -DPIC works... yes
 330 checking if gcc static flag -static works... no
 331 checking if gcc supports -c -o file.o... yes
 332 checking if gcc supports -c -o file.o... (cached) yes
 333 checking whether the gcc linker (/bin/ld -m elf_x86_64) supports shared libraries... yes
 334 checking whether -lc should be explicitly linked in... no
 335 checking dynamic linker characteristics... GNU/Linux ld.so
 336 checking how to hardcode library paths into programs... immediate
 337 checking whether stripping libraries is possible... yes
 338 checking if libtool supports shared libraries... yes
 339 checking whether to build shared libraries... yes
 340 checking whether to build static libraries... yes
 341 checking for gcc... (cached) gcc
 342 checking whether we are using the GNU C compiler... (cached) yes
 343 checking whether gcc accepts -g... (cached) yes
 344 checking for gcc option to accept ISO C89... (cached) none needed
 345 checking whether gcc understands -c and -o together... (cached) yes
 346 checking dependency style of gcc... (cached) gcc3
 347 checking whether ln -s works... yes
 348 checking for pkg-config... /bin/pkg-config
 349 checking pkg-config is at least version 0.9.0... yes
 350 checking dependency style of gcc... gcc3
 351 checking whether to build with code coverage support... no
 352 checking how to create a pax tar archive... gnutar
 353 checking zfs author... OpenZFS
 354 checking zfs license... CDDL
 355 checking whether NLS is requested... yes
 356 checking for msgfmt... /bin/msgfmt
 357 checking for gmsgfmt... /bin/msgfmt
 358 checking for xgettext... /bin/xgettext
 359 checking for msgmerge... /bin/msgmerge
 360 checking for ld... /bin/ld -m elf_x86_64
 361 checking if the linker (/bin/ld -m elf_x86_64) is GNU ld... yes
 362 checking for shared library run path origin... done
 363 checking 32-bit host C ABI... no
 364 checking for the common suffixes of directories in the library search path... lib64,lib64
 365 checking zfs config... kernel
 366 checking the number of available CPUs... 64
 367 checking whether gcc supports -Wno-unused-but-set-variable... yes
 368 checking whether gcc supports -Wno-bool-compare... yes
 369 checking whether gcc supports -Wframe-larger-than=<size>... yes
 370 checking whether gcc supports -Wno-format-truncation... yes
 371 checking whether gcc supports -Wno-format-zero-length... yes
 372 checking whether gcc supports -fno-omit-frame-pointer... yes
 373 checking whether gcc supports -fno-ipa-sra... yes
 374 checking whether to build with -fsanitize=address support... no
 375 checking whether host toolchain supports SSE... yes
 376 checking whether host toolchain supports SSE2... yes
 377 checking whether host toolchain supports SSE3... yes
 378 checking whether host toolchain supports SSSE3... yes
 379 checking whether host toolchain supports SSE4.1... yes
 380 checking whether host toolchain supports SSE4.2... yes
 381 checking whether host toolchain supports AVX... yes
 382 checking whether host toolchain supports AVX2... yes
 383 checking whether host toolchain supports AVX512F... yes
 384 checking whether host toolchain supports AVX512CD... yes
 385 checking whether host toolchain supports AVX512DQ... yes
 386 checking whether host toolchain supports AVX512BW... yes
 387 checking whether host toolchain supports AVX512IFMA... yes
 388 checking whether host toolchain supports AVX512VBMI... yes
 389 checking whether host toolchain supports AVX512PF... yes
 390 checking whether host toolchain supports AVX512ER... yes
 391 checking whether host toolchain supports AVX512VL... yes
 392 checking whether host toolchain supports AES... yes
 393 checking whether host toolchain supports PCLMULQDQ... yes
 394 checking whether host toolchain supports MOVBE... yes
 395 checking for system type (linux-gnu)... Linux
 396 checking for python3... python3
 397 checking for python version... 3.9
 398 checking for python platform... linux
 399 checking for python script directory... ${prefix}/lib/python3.9/site-packages
 400 checking for python extension module directory... ${exec_prefix}/lib64/python3.9/site-packages
 401 checking for python3.9... /bin/python3.9
 402 checking for a version of Python >= '2.1.0'... yes
 403 checking for a version of Python >= '3.4.0'... yes
 404 checking for the distutils Python package... yes
 405 checking for Python include path... -I/usr/include/python3.9
 406 checking for Python library path... -L/usr/lib64 -lpython3.9
 407 checking for Python site-packages path... /usr/lib/python3.9/site-packages
 408 checking python extra libraries... -lcrypt -lpthread -ldl  -lutil -lm -lm
 409 checking python extra linking flags... -Xlinker -export-dynamic
 410 checking consistency of all components of python development environment... no
 411 checking whether to enable pyzfs: ... no
 412 checking for sed --in-place... --in-place
 413 checking for cppcheck... no
 414 checking for shellcheck... no
 415 checking for checkbashisms... no
 416 checking kernel source directory... /lib/modules/5.13.8-200.fc34.x86_64/build
 417 checking kernel build directory... /lib/modules/5.13.8-200.fc34.x86_64/build
 418 checking kernel source version... 5.13.8-200.fc34.x86_64
 419 checking kernel file name for module symbols... Module.symvers
 420 checking whether modules can be built... yes
 421 checking for kernel config option compatibility... done
 422 checking whether kernel was built with 16K or larger stacks... yes
 423 checking whether mutex_lock() is GPL-only... no
 424 checking whether CONFIG_TRIM_UNUSED_KSYM is disabled... yes
 425 checking whether CONFIG_ZLIB_INFLATE is defined... yes
 426 checking whether CONFIG_ZLIB_DEFLATE is defined... yes
 427 checking whether fpu headers are available... asm/fpu/api.h
 428 checking whether objtool header is available... linux/objtool.h
 429 checking whether wait_queue_entry_t exists... yes
 430 checking whether /dev/zfs minor is available... 249
 431 checking whether DECLARE_EVENT_CLASS() is available... no
 432 checking for available kernel interfaces... done
 433 checking whether access_ok() has 'type' parameter... no
 434 checking whether global_node_page_state() exists... yes
 435 checking whether global_zone_page_state() exists... yes
 436 checking whether enum node_stat_item contains NR_FILE_PAGES... yes
 437 checking whether enum node_stat_item contains NR_INACTIVE_ANON... yes
 438 checking whether enum node_stat_item contains NR_INACTIVE_FILE... yes
 439 checking whether enum zone_stat_item contains NR_FILE_PAGES... no
 440 checking whether enum zone_stat_item contains NR_INACTIVE_ANON... no
 441 checking whether enum zone_stat_item contains NR_INACTIVE_FILE... no
 442 checking whether global_page_state enums are sane... yes
 443 checking whether compile-time stack validation (objtool) is available... yes
 444 checking whether STACK_FRAME_NON_STANDARD is defined... yes
 445 checking whether PDE_DATA() is available... yes
 446 checking whether fops->fallocate() exists... yes
 447 checking whether zlib_deflate_workspacesize() wants 2 args... yes
 448 checking whether struct rw_semaphore has member activity... no
 449 checking whether struct rw_semaphore has atomic_long_t member count... yes
 450 checking whether header linux/sched/rt.h exists... yes
 451 checking whether header linux/sched/signal.h exists... yes
 452 checking whether io_schedule_timeout() is available... yes
 453 checking whether usleep_range() is available... yes
 454 checking whether kmem_cache_create_usercopy() exists... yes
 455 checking whether kvmalloc(ptr, flags) is available... yes
 456 checking whether __vmalloc(ptr, flags, pageflags) is available... no
 457 checking whether wait_on_bit() takes an action... no
 458 checking whether wq_head->head and wq_entry->entry exist... yes
 459 checking whether timestamp_truncate() exists... yes
 460 checking whether inode->i_*time's are timespec64... yes
 461 checking whether inode_lock_shared() exists... yes
 462 checking whether group_info->gid exists... yes
 463 checking whether kernel_write() takes loff_t pointer... yes
 464 checking whether kernel_read() takes loff_t pointer... yes
 465 checking whether timer_setup() is available... yes
 466 checking whether timer function expects timer_list... yes
 467 checking whether struct timer_list has flags... yes
 468 checking whether super_block->s_user_ns exists... yes
 469 checking whether proc_ops structure exists... yes
 470 checking whether bops->check_events() exists... yes
 471 checking whether bops->release() is void... yes
 472 checking whether bops->revalidate_disk() exists... no
 473 checking whether REQ_FAILFAST_MASK is defined... yes
 474 checking whether REQ_DISCARD is defined... no
 475 checking whether REQ_FLUSH is defined... no
 476 checking whether REQ_PREFLUSH is defined... yes
 477 checking whether REQ_OP_DISCARD is defined... yes
 478 checking whether REQ_OP_SECURE_ERASE is defined... yes
 479 checking whether REQ_OP_FLUSH is defined... yes
 480 checking whether bio->bi_opf is defined... yes
 481 checking whether bio_set_op_attrs is available... yes
 482 checking whether bio_set_dev() is available... yes
 483 checking whether bio_set_dev() is GPL-only... yes
 484 checking whether bio_end_io_t wants 1 arg... yes
 485 checking whether bio->bi_status exists... yes
 486 checking whether bio has bi_iter... yes
 487 checking whether submit_bio() wants 1 arg... yes
 488 checking whether current->bio_list exists... yes
 489 checking whether blkg_tryget() is available... yes
 490 checking whether blkg_tryget() is GPL-only... yes
 491 checking whether bio->bi_bdev->bd_disk exists... yes
 492 checking whether blkdev_get_by_path() exists... yes
 493 checking whether blkdev_put() exists... yes
 494 checking whether blkdev_reread_part() exists... no
 495 checking whether invalidate_bdev() exists... yes
 496 checking whether lookup_bdev() wants dev_t arg... yes
 497 checking whether bdev_logical_block_size() is available... yes
 498 checking whether bdev_physical_block_size() is available... yes
 499 checking whether check_disk_change() exists... no
 500 checking whether bdev_disk_changed() exists... yes
 501 checking whether bdev_whole() is available... yes
 502 checking whether struct blk_plug is available... yes
 503 checking whether blk_queue bdi is dynamic... yes
 504 checking whether blk_queue_discard() is available... yes
 505 checking whether blk_queue_secure_erase() is available... yes
 506 checking whether blk_queue_flag_set() exists... yes
 507 checking whether blk_queue_flag_clear() exists... yes
 508 checking whether blk_queue_flush() is available... no
 509 checking whether blk_queue_write_cache() exists... yes
 510 checking whether blk_queue_write_cache() is GPL-only... yes
 511 checking whether blk_queue_max_hw_sectors() is available... yes
 512 checking whether blk_queue_max_segments() is available... yes
 513 checking whether revalidate_disk_size() is available... no
 514 checking whether revalidate_disk() is available... no
 515 checking whether get_disk_ro() is available... yes
 516 checking whether generic_readlink is global... no
 517 checking whether ql->discard_granularity is available... yes
 518 checking whether inode_owner_or_capable() exists... no
 519 checking whether inode_owner_or_capable() takes user_ns... yes
 520 checking whether super_block uses const struct xattr_handler... yes
 521 checking whether xattr_handler has name... yes
 522 checking whether xattr_handler->get() wants dentry and inode... yes
 523 checking whether xattr_handler->set() wants dentry, inode, and user_namespace... yes
 524 checking whether xattr_handler->list() wants simple... yes
 525 checking whether posix_acl_from_xattr() needs user_ns... yes
 526 checking whether generic_setxattr() exists... no
 527 checking whether posix_acl_release() is available... yes
 528 checking whether posix_acl_release() is GPL-only... yes
 529 checking whether set_cached_acl() is usable... yes
 530 checking whether __posix_acl_chmod exists... yes
 531 checking whether posix_acl_equiv_mode() wants umode_t... yes
 532 checking whether posix_acl_valid() wants user namespace... yes
 533 checking whether iops->get_acl() exists... yes
 534 checking whether iops->set_acl() exists... yes
 535 checking whether uncached_acl_sentinel() exists... yes
 536 checking whether posix_acl has refcount_t... yes
 537 checking whether iops->getattr() takes user_namespace... yes
 538 checking whether inode_set_flags() exists... yes
 539 checking whether inode_set_iversion() exists... yes
 540 checking whether sops->show_options() wants dentry... yes
 541 checking whether file_inode() is available... yes
 542 checking whether file_dentry() is available... yes
 543 checking whether fops->fsync() wants no dentry... no
 544 checking whether fops->fsync() wants range... range
 545 checking whether fops->aio_fsync() exists... no
 546 checking whether sops->evict_inode() exists... yes
 547 checking whether sops->dirty_inode() wants flags... yes
 548 checking whether super_block has s_shrink... yes
 549 checking whether shrink_control has nid... yes
 550 checking whether new 2-argument shrinker exists... no
 551 checking whether ->count_objects callback exists... yes
 552 checking whether struct shrink_control exists... yes
 553 checking whether iops->mkdir() takes struct user_namespace*... yes
 554 checking whether iops->lookup() passes flags... yes
 555 checking whether iops->create() takes struct user_namespace*... yes
 556 checking whether iops->get_link() passes delayed... yes
 557 checking whether i_op->tmpfile() exists... yes
 558 checking whether dops->d_automount() exists... yes
 559 checking whether eops->encode_fh() wants inode... yes
 560 checking whether eops->commit_metadata() exists... yes
 561 checking whether clear_inode() is available... yes
 562 checking whether setattr_prepare() is available and accepts struct user_namespace*... yes
 563 checking whether insert_inode_locked() is available... yes
 564 checking whether d_make_root() is available... yes
 565 checking whether d_obtain_alias() is available... yes
 566 checking whether d_prune_aliases() is available... yes
 567 checking whether d_set_d_op() is available... yes
 568 checking whether dops->d_revalidate() takes struct nameidata... no
 569 checking whether dentry uses const struct dentry_operations... yes
 570 checking whether super_block has s_d_op... yes
 571 checking whether truncate_setsize() is available... yes
 572 checking whether security_inode_init_security wants callback... yes
 573 checking whether fst->mount() exists... yes
 574 checking whether super_setup_bdi_name() exists... yes
 575 checking whether set_nlink() is available... yes
 576 checking whether sget() wants 5 args... yes
 577 checking whether lseek_execute() is available... no
 578 checking whether vfs_getattr() wants 4 args... yes
 579 checking whether vfs_getattr() wants 2 args... no
 580 checking whether vfs_getattr() wants 3 args... no
 581 checking whether vfs_fsync() wants 2 args... yes
 582 checking whether fops->iterate_shared() is available... yes
 583 checking whether aops->direct_IO() uses iov_iter... yes
 584 checking whether fops->read/write_iter() are available... yes
 585 checking whether new_sync_read/write() are available... no
 586 checking whether generic_write_checks() takes kiocb... yes
 587 checking whether iov_iter types are available... yes
 588 checking whether iov_iter_advance() is available... yes
 589 checking whether iov_iter_revert() is available... yes
 590 checking whether iov_iter_fault_in_readable() is available... yes
 591 checking whether iov_iter_count() is available... yes
 592 checking whether copy_to_iter() is available... yes
 593 checking whether copy_from_iter() is available... yes
 594 checking whether kmap_atomic wants 1 args... yes
 595 checking whether follow_down_one() is available... yes
 596 checking whether submit_bio is member of struct block_device_operations... yes
 597 checking whether generic disk_*_io_acct() are available... yes
 598 checking whether kernel fpu is available... internal
 599 checking whether kernel defines fmode_t... yes
 600 checking whether kuid_t/kgid_t is available... yes
 601 checking whether i_(uid|gid)_(read|write) exist... yes
 602 checking whether module_param_call() is hardened... yes
 603 checking whether iops->rename() takes struct user_namespace*... yes
 604 checking whether current_time() exists... yes
 605 checking whether ns_capable exists... yes
 606 checking whether has_capability() is available... yes
 607 checking whether cred_t->user_ns exists... yes
 608 checking whether kuid_has_mapping/kgid_has_mapping exist... yes
 609 checking whether in_compat_syscall() is available... yes
 610 checking whether ktime_get_coarse_real_ts64() exists... yes
 611 checking whether ktime_get_raw_ts64() exists... yes
 612 checking whether totalram_pages() exists... yes
 613 checking whether totalhigh_pages() exists... yes
 614 checking whether kstrtoul() exists... yes
 615 checking whether percpu_counter_init() wants gfp_t... yes
 616 checking whether percpu_counter_add_batch() is defined... yes
 617 checking whether is inside percpu_ref.data... yes
 618 checking whether CPU hotplug APIs exist... yes
 619 checking whether generic_fillattr requires struct user_namespace*... yes
 620 checking whether iops->mknod() takes struct user_namespace*... yes
 621 checking whether iops->symlink() takes struct user_namespace*... yes
 622 checking whether bio_max_segs() exists... yes
 623 checking whether signal_stop() exists... yes
 624 checking whether kernel_siginfo_t tyepedef exists... yes
 625 checking whether set_special_state() exists... yes
 626 checking os distribution... fedora
 627 checking default package type... rpm
 628 checking default init directory... ${prefix}/etc/init.d
 629 checking default init script type and shell... fedora:/bin/sh
 630 checking default nfs server init script... nfs
 631 checking default init config directory... /etc/sysconfig
 632 checking whether initramfs-tools is available... no
 633 checking whether rpm is available... yes (4.16.1.3)
 634 checking whether rpmbuild is available... yes (4.16.1.3)
 635 checking whether spec files are available... yes (rpm/generic/*.spec.in)
 636 checking whether dpkg is available... no
 637 checking whether dpkg-buildpackage is available... no
 638 checking whether alien is available... no
 639 checking whether assertion support will be enabled... no
 640 checking whether debuginfo support will be forced... no
 641 checking whether basic kmem accounting is enabled... no
 642 checking whether detailed kmem tracking is enabled... no
 643 checking whether FreeBSD kernel INVARIANTS checks are enabled... no
 644 checking that generated files are newer than configure... done
 645 configure: creating ./config.status
 646 config.status: creating Makefile
 647 config.status: creating cmd/Makefile
 648 config.status: creating cmd/arc_summary/Makefile
 649 config.status: creating cmd/arcstat/Makefile
 650 config.status: creating cmd/dbufstat/Makefile
 651 config.status: creating cmd/fsck_zfs/Makefile
 652 config.status: creating cmd/mount_zfs/Makefile
 653 config.status: creating cmd/raidz_test/Makefile
 654 config.status: creating cmd/vdev_id/Makefile
 655 config.status: creating cmd/zdb/Makefile
 656 config.status: creating cmd/zed/Makefile
 657 config.status: creating cmd/zed/zed.d/Makefile
 658 config.status: creating cmd/zfs/Makefile
 659 config.status: creating cmd/zfs_ids_to_path/Makefile
 660 config.status: creating cmd/zgenhostid/Makefile
 661 config.status: creating cmd/zhack/Makefile
 662 config.status: creating cmd/zinject/Makefile
 663 config.status: creating cmd/zpool/Makefile
 664 config.status: creating cmd/zstream/Makefile
 665 config.status: creating cmd/ztest/Makefile
 666 config.status: creating cmd/zvol_id/Makefile
 667 config.status: creating cmd/zvol_wait/Makefile
 668 config.status: creating cmd/zpool_influxdb/Makefile
 669 config.status: creating contrib/Makefile
 670 config.status: creating contrib/bash_completion.d/Makefile
 671 config.status: creating contrib/bpftrace/Makefile
 672 config.status: creating contrib/dracut/02zfsexpandknowledge/Makefile
 673 config.status: creating contrib/dracut/90zfs/Makefile
 674 config.status: creating contrib/dracut/Makefile
 675 config.status: creating contrib/initramfs/Makefile
 676 config.status: creating contrib/initramfs/conf.d/Makefile
 677 config.status: creating contrib/initramfs/conf-hooks.d/Makefile
 678 config.status: creating contrib/initramfs/hooks/Makefile
 679 config.status: creating contrib/initramfs/scripts/Makefile
 680 config.status: creating contrib/initramfs/scripts/local-top/Makefile
 681 config.status: creating contrib/pam_zfs_key/Makefile
 682 config.status: creating contrib/pyzfs/Makefile
 683 config.status: creating contrib/pyzfs/setup.py
 684 config.status: creating contrib/zcp/Makefile
 685 config.status: creating etc/Makefile
 686 config.status: creating etc/default/Makefile
 687 config.status: creating etc/init.d/Makefile
 688 config.status: creating etc/modules-load.d/Makefile
 689 config.status: creating etc/sudoers.d/Makefile
 690 config.status: creating etc/systemd/Makefile
 691 config.status: creating etc/systemd/system-generators/Makefile
 692 config.status: creating etc/systemd/system/Makefile
 693 config.status: creating etc/zfs/Makefile
 694 config.status: creating include/Makefile
 695 config.status: creating include/os/Makefile
 696 config.status: creating include/os/freebsd/Makefile
 697 config.status: creating include/os/freebsd/linux/Makefile
 698 config.status: creating include/os/freebsd/spl/Makefile
 699 config.status: creating include/os/freebsd/spl/acl/Makefile
 700 config.status: creating include/os/freebsd/spl/rpc/Makefile
 701 config.status: creating include/os/freebsd/spl/sys/Makefile
 702 config.status: creating include/os/freebsd/zfs/Makefile
 703 config.status: creating include/os/freebsd/zfs/sys/Makefile
 704 config.status: creating include/os/linux/Makefile
 705 config.status: creating include/os/linux/kernel/Makefile
 706 config.status: creating include/os/linux/kernel/linux/Makefile
 707 config.status: creating include/os/linux/spl/Makefile
 708 config.status: creating include/os/linux/spl/rpc/Makefile
 709 config.status: creating include/os/linux/spl/sys/Makefile
 710 config.status: creating include/os/linux/zfs/Makefile
 711 config.status: creating include/os/linux/zfs/sys/Makefile
 712 config.status: creating include/sys/Makefile
 713 config.status: creating include/sys/crypto/Makefile
 714 config.status: creating include/sys/fm/Makefile
 715 config.status: creating include/sys/fm/fs/Makefile
 716 config.status: creating include/sys/fs/Makefile
 717 config.status: creating include/sys/lua/Makefile
 718 config.status: creating include/sys/sysevent/Makefile
 719 config.status: creating include/sys/zstd/Makefile
 720 config.status: creating lib/Makefile
 721 config.status: creating lib/libavl/Makefile
 722 config.status: creating lib/libefi/Makefile
 723 config.status: creating lib/libicp/Makefile
 724 config.status: creating lib/libnvpair/Makefile
 725 config.status: creating lib/libshare/Makefile
 726 config.status: creating lib/libspl/Makefile
 727 config.status: creating lib/libspl/include/Makefile
 728 config.status: creating lib/libspl/include/ia32/Makefile
 729 config.status: creating lib/libspl/include/ia32/sys/Makefile
 730 config.status: creating lib/libspl/include/os/Makefile
 731 config.status: creating lib/libspl/include/os/freebsd/Makefile
 732 config.status: creating lib/libspl/include/os/freebsd/sys/Makefile
 733 config.status: creating lib/libspl/include/os/linux/Makefile
 734 config.status: creating lib/libspl/include/os/linux/sys/Makefile
 735 config.status: creating lib/libspl/include/rpc/Makefile
 736 config.status: creating lib/libspl/include/sys/Makefile
 737 config.status: creating lib/libspl/include/sys/dktp/Makefile
 738 config.status: creating lib/libspl/include/util/Makefile
 739 config.status: creating lib/libtpool/Makefile
 740 config.status: creating lib/libunicode/Makefile
 741 config.status: creating lib/libuutil/Makefile
 742 config.status: creating lib/libzfs/Makefile
 743 config.status: creating lib/libzfs/libzfs.pc
 744 config.status: creating lib/libzfsbootenv/Makefile
 745 config.status: creating lib/libzfsbootenv/libzfsbootenv.pc
 746 config.status: creating lib/libzfs_core/Makefile
 747 config.status: creating lib/libzfs_core/libzfs_core.pc
 748 config.status: creating lib/libzpool/Makefile
 749 config.status: creating lib/libzstd/Makefile
 750 config.status: creating lib/libzutil/Makefile
 751 config.status: creating man/Makefile
 752 config.status: creating module/Kbuild
 753 config.status: creating module/Makefile
 754 config.status: creating module/avl/Makefile
 755 config.status: creating module/icp/Makefile
 756 config.status: creating module/lua/Makefile
 757 config.status: creating module/nvpair/Makefile
 758 config.status: creating module/os/linux/spl/Makefile
 759 config.status: creating module/os/linux/zfs/Makefile
 760 config.status: creating module/spl/Makefile
 761 config.status: creating module/unicode/Makefile
 762 config.status: creating module/zcommon/Makefile
 763 config.status: creating module/zfs/Makefile
 764 config.status: creating module/zstd/Makefile
 765 config.status: creating rpm/Makefile
 766 config.status: creating rpm/generic/Makefile
 767 config.status: creating rpm/generic/zfs-dkms.spec
 768 config.status: creating rpm/generic/zfs-kmod.spec
 769 config.status: creating rpm/generic/zfs.spec
 770 config.status: creating rpm/redhat/Makefile
 771 config.status: creating rpm/redhat/zfs-dkms.spec
 772 config.status: creating rpm/redhat/zfs-kmod.spec
 773 config.status: creating rpm/redhat/zfs.spec
 774 config.status: creating scripts/Makefile
 775 config.status: creating tests/Makefile
 776 config.status: creating tests/runfiles/Makefile
 777 config.status: creating tests/test-runner/Makefile
 778 config.status: creating tests/test-runner/bin/Makefile
 779 config.status: creating tests/test-runner/include/Makefile
 780 config.status: creating tests/test-runner/man/Makefile
 781 config.status: creating tests/zfs-tests/Makefile
 782 config.status: creating tests/zfs-tests/callbacks/Makefile
 783 config.status: creating tests/zfs-tests/cmd/Makefile
 784 config.status: creating tests/zfs-tests/cmd/badsend/Makefile
 785 config.status: creating tests/zfs-tests/cmd/btree_test/Makefile
 786 config.status: creating tests/zfs-tests/cmd/chg_usr_exec/Makefile
 787 config.status: creating tests/zfs-tests/cmd/devname2devid/Makefile
 788 config.status: creating tests/zfs-tests/cmd/draid/Makefile
 789 config.status: creating tests/zfs-tests/cmd/dir_rd_update/Makefile
 790 config.status: creating tests/zfs-tests/cmd/file_check/Makefile
 791 config.status: creating tests/zfs-tests/cmd/file_trunc/Makefile
 792 config.status: creating tests/zfs-tests/cmd/file_write/Makefile
 793 config.status: creating tests/zfs-tests/cmd/get_diff/Makefile
 794 config.status: creating tests/zfs-tests/cmd/largest_file/Makefile
 795 config.status: creating tests/zfs-tests/cmd/libzfs_input_check/Makefile
 796 config.status: creating tests/zfs-tests/cmd/mkbusy/Makefile
 797 config.status: creating tests/zfs-tests/cmd/mkfile/Makefile
 798 config.status: creating tests/zfs-tests/cmd/mkfiles/Makefile
 799 config.status: creating tests/zfs-tests/cmd/mktree/Makefile
 800 config.status: creating tests/zfs-tests/cmd/mmap_exec/Makefile
 801 config.status: creating tests/zfs-tests/cmd/mmap_libaio/Makefile
 802 config.status: creating tests/zfs-tests/cmd/mmapwrite/Makefile
 803 config.status: creating tests/zfs-tests/cmd/nvlist_to_lua/Makefile
 804 config.status: creating tests/zfs-tests/cmd/randfree_file/Makefile
 805 config.status: creating tests/zfs-tests/cmd/randwritecomp/Makefile
 806 config.status: creating tests/zfs-tests/cmd/readmmap/Makefile
 807 config.status: creating tests/zfs-tests/cmd/rename_dir/Makefile
 808 config.status: creating tests/zfs-tests/cmd/rm_lnkcnt_zero_file/Makefile
 809 config.status: creating tests/zfs-tests/cmd/send_doall/Makefile
 810 config.status: creating tests/zfs-tests/cmd/stride_dd/Makefile
 811 config.status: creating tests/zfs-tests/cmd/threadsappend/Makefile
 812 config.status: creating tests/zfs-tests/cmd/user_ns_exec/Makefile
 813 config.status: creating tests/zfs-tests/cmd/xattrtest/Makefile
 814 config.status: creating tests/zfs-tests/include/Makefile
 815 config.status: creating tests/zfs-tests/tests/Makefile
 816 config.status: creating tests/zfs-tests/tests/functional/Makefile
 817 config.status: creating tests/zfs-tests/tests/functional/acl/Makefile
 818 config.status: creating tests/zfs-tests/tests/functional/acl/off/Makefile
 819 config.status: creating tests/zfs-tests/tests/functional/acl/posix/Makefile
 820 config.status: creating tests/zfs-tests/tests/functional/acl/posix-sa/Makefile
 821 config.status: creating tests/zfs-tests/tests/functional/alloc_class/Makefile
 822 config.status: creating tests/zfs-tests/tests/functional/arc/Makefile
 823 config.status: creating tests/zfs-tests/tests/functional/atime/Makefile
 824 config.status: creating tests/zfs-tests/tests/functional/bootfs/Makefile
 825 config.status: creating tests/zfs-tests/tests/functional/btree/Makefile
 826 config.status: creating tests/zfs-tests/tests/functional/cache/Makefile
 827 config.status: creating tests/zfs-tests/tests/functional/cachefile/Makefile
 828 config.status: creating tests/zfs-tests/tests/functional/casenorm/Makefile
 829 config.status: creating tests/zfs-tests/tests/functional/channel_program/Makefile
 830 config.status: creating tests/zfs-tests/tests/functional/channel_program/lua_core/Makefile
 831 config.status: creating tests/zfs-tests/tests/functional/channel_program/synctask_core/Makefile
 832 config.status: creating tests/zfs-tests/tests/functional/chattr/Makefile
 833 config.status: creating tests/zfs-tests/tests/functional/checksum/Makefile
 834 config.status: creating tests/zfs-tests/tests/functional/clean_mirror/Makefile
 835 config.status: creating tests/zfs-tests/tests/functional/cli_root/Makefile
 836 config.status: creating tests/zfs-tests/tests/functional/cli_root/zdb/Makefile
 837 config.status: creating tests/zfs-tests/tests/functional/cli_root/zfs/Makefile
 838 config.status: creating tests/zfs-tests/tests/functional/cli_root/zfs_bookmark/Makefile
 839 config.status: creating tests/zfs-tests/tests/functional/cli_root/zfs_change-key/Makefile
 840 config.status: creating tests/zfs-tests/tests/functional/cli_root/zfs_clone/Makefile
 841 config.status: creating tests/zfs-tests/tests/functional/cli_root/zfs_copies/Makefile
 842 config.status: creating tests/zfs-tests/tests/functional/cli_root/zfs_create/Makefile
 843 config.status: creating tests/zfs-tests/tests/functional/cli_root/zfs_destroy/Makefile
 844 config.status: creating tests/zfs-tests/tests/functional/cli_root/zfs_diff/Makefile
 845 config.status: creating tests/zfs-tests/tests/functional/cli_root/zfs_get/Makefile
 846 config.status: creating tests/zfs-tests/tests/functional/cli_root/zfs_ids_to_path/Makefile
 847 config.status: creating tests/zfs-tests/tests/functional/cli_root/zfs_inherit/Makefile
 848 config.status: creating tests/zfs-tests/tests/functional/cli_root/zfs_jail/Makefile
 849 config.status: creating tests/zfs-tests/tests/functional/cli_root/zfs_load-key/Makefile
 850 config.status: creating tests/zfs-tests/tests/functional/cli_root/zfs_mount/Makefile
 851 config.status: creating tests/zfs-tests/tests/functional/cli_root/zfs_program/Makefile
 852 config.status: creating tests/zfs-tests/tests/functional/cli_root/zfs_promote/Makefile
 853 config.status: creating tests/zfs-tests/tests/functional/cli_root/zfs_property/Makefile
 854 config.status: creating tests/zfs-tests/tests/functional/cli_root/zfs_receive/Makefile
 855 config.status: creating tests/zfs-tests/tests/functional/cli_root/zfs_rename/Makefile
 856 config.status: creating tests/zfs-tests/tests/functional/cli_root/zfs_reservation/Makefile
 857 config.status: creating tests/zfs-tests/tests/functional/cli_root/zfs_rollback/Makefile
 858 config.status: creating tests/zfs-tests/tests/functional/cli_root/zfs_send/Makefile
 859 config.status: creating tests/zfs-tests/tests/functional/cli_root/zfs_set/Makefile
 860 config.status: creating tests/zfs-tests/tests/functional/cli_root/zfs_share/Makefile
 861 config.status: creating tests/zfs-tests/tests/functional/cli_root/zfs_snapshot/Makefile
 862 config.status: creating tests/zfs-tests/tests/functional/cli_root/zfs_sysfs/Makefile
 863 config.status: creating tests/zfs-tests/tests/functional/cli_root/zfs_unload-key/Makefile
 864 config.status: creating tests/zfs-tests/tests/functional/cli_root/zfs_unmount/Makefile
 865 config.status: creating tests/zfs-tests/tests/functional/cli_root/zfs_unshare/Makefile
 866 config.status: creating tests/zfs-tests/tests/functional/cli_root/zfs_upgrade/Makefile
 867 config.status: creating tests/zfs-tests/tests/functional/cli_root/zfs_wait/Makefile
 868 config.status: creating tests/zfs-tests/tests/functional/cli_root/zpool/Makefile
 869 config.status: creating tests/zfs-tests/tests/functional/cli_root/zpool_add/Makefile
 870 config.status: creating tests/zfs-tests/tests/functional/cli_root/zpool_attach/Makefile
 871 config.status: creating tests/zfs-tests/tests/functional/cli_root/zpool_clear/Makefile
 872 config.status: creating tests/zfs-tests/tests/functional/cli_root/zpool_create/Makefile
 873 config.status: creating tests/zfs-tests/tests/functional/cli_root/zpool_destroy/Makefile
 874 config.status: creating tests/zfs-tests/tests/functional/cli_root/zpool_detach/Makefile
 875 config.status: creating tests/zfs-tests/tests/functional/cli_root/zpool_events/Makefile
 876 config.status: creating tests/zfs-tests/tests/functional/cli_root/zpool_expand/Makefile
 877 config.status: creating tests/zfs-tests/tests/functional/cli_root/zpool_export/Makefile
 878 config.status: creating tests/zfs-tests/tests/functional/cli_root/zpool_get/Makefile
 879 config.status: creating tests/zfs-tests/tests/functional/cli_root/zpool_history/Makefile
 880 config.status: creating tests/zfs-tests/tests/functional/cli_root/zpool_import/Makefile
 881 config.status: creating tests/zfs-tests/tests/functional/cli_root/zpool_import/blockfiles/Makefile
 882 config.status: creating tests/zfs-tests/tests/functional/cli_root/zpool_initialize/Makefile
 883 config.status: creating tests/zfs-tests/tests/functional/cli_root/zpool_labelclear/Makefile
 884 config.status: creating tests/zfs-tests/tests/functional/cli_root/zpool_offline/Makefile
 885 config.status: creating tests/zfs-tests/tests/functional/cli_root/zpool_online/Makefile
 886 config.status: creating tests/zfs-tests/tests/functional/cli_root/zpool_remove/Makefile
 887 config.status: creating tests/zfs-tests/tests/functional/cli_root/zpool_reopen/Makefile
 888 config.status: creating tests/zfs-tests/tests/functional/cli_root/zpool_replace/Makefile
 889 config.status: creating tests/zfs-tests/tests/functional/cli_root/zpool_resilver/Makefile
 890 config.status: creating tests/zfs-tests/tests/functional/cli_root/zpool_scrub/Makefile
 891 config.status: creating tests/zfs-tests/tests/functional/cli_root/zpool_set/Makefile
 892 config.status: creating tests/zfs-tests/tests/functional/cli_root/zpool_split/Makefile
 893 config.status: creating tests/zfs-tests/tests/functional/cli_root/zpool_status/Makefile
 894 config.status: creating tests/zfs-tests/tests/functional/cli_root/zpool_sync/Makefile
 895 config.status: creating tests/zfs-tests/tests/functional/cli_root/zpool_trim/Makefile
 896 config.status: creating tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/Makefile
 897 config.status: creating tests/zfs-tests/tests/functional/cli_root/zpool_upgrade/blockfiles/Makefile
 898 config.status: creating tests/zfs-tests/tests/functional/cli_root/zpool_wait/Makefile
 899 config.status: creating tests/zfs-tests/tests/functional/cli_root/zpool_wait/scan/Makefile
 900 config.status: creating tests/zfs-tests/tests/functional/cli_user/Makefile
 901 config.status: creating tests/zfs-tests/tests/functional/cli_user/misc/Makefile
 902 config.status: creating tests/zfs-tests/tests/functional/cli_user/zfs_list/Makefile
 903 config.status: creating tests/zfs-tests/tests/functional/cli_user/zpool_iostat/Makefile
 904 config.status: creating tests/zfs-tests/tests/functional/cli_user/zpool_list/Makefile
 905 config.status: creating tests/zfs-tests/tests/functional/cli_user/zpool_status/Makefile
 906 config.status: creating tests/zfs-tests/tests/functional/compression/Makefile
 907 config.status: creating tests/zfs-tests/tests/functional/cp_files/Makefile
 908 config.status: creating tests/zfs-tests/tests/functional/ctime/Makefile
 909 config.status: creating tests/zfs-tests/tests/functional/deadman/Makefile
 910 config.status: creating tests/zfs-tests/tests/functional/delegate/Makefile
 911 config.status: creating tests/zfs-tests/tests/functional/devices/Makefile
 912 config.status: creating tests/zfs-tests/tests/functional/events/Makefile
 913 config.status: creating tests/zfs-tests/tests/functional/exec/Makefile
 914 config.status: creating tests/zfs-tests/tests/functional/fallocate/Makefile
 915 config.status: creating tests/zfs-tests/tests/functional/fault/Makefile
 916 config.status: creating tests/zfs-tests/tests/functional/features/Makefile
 917 config.status: creating tests/zfs-tests/tests/functional/features/async_destroy/Makefile
 918 config.status: creating tests/zfs-tests/tests/functional/features/large_dnode/Makefile
 919 config.status: creating tests/zfs-tests/tests/functional/grow/Makefile
 920 config.status: creating tests/zfs-tests/tests/functional/history/Makefile
 921 config.status: creating tests/zfs-tests/tests/functional/hkdf/Makefile
 922 config.status: creating tests/zfs-tests/tests/functional/inheritance/Makefile
 923 config.status: creating tests/zfs-tests/tests/functional/inuse/Makefile
 924 config.status: creating tests/zfs-tests/tests/functional/io/Makefile
 925 config.status: creating tests/zfs-tests/tests/functional/l2arc/Makefile
 926 config.status: creating tests/zfs-tests/tests/functional/large_files/Makefile
 927 config.status: creating tests/zfs-tests/tests/functional/largest_pool/Makefile
 928 config.status: creating tests/zfs-tests/tests/functional/libzfs/Makefile
 929 config.status: creating tests/zfs-tests/tests/functional/limits/Makefile
 930 config.status: creating tests/zfs-tests/tests/functional/link_count/Makefile
 931 config.status: creating tests/zfs-tests/tests/functional/log_spacemap/Makefile
 932 config.status: creating tests/zfs-tests/tests/functional/migration/Makefile
 933 config.status: creating tests/zfs-tests/tests/functional/mmap/Makefile
 934 config.status: creating tests/zfs-tests/tests/functional/mmp/Makefile
 935 config.status: creating tests/zfs-tests/tests/functional/mount/Makefile
 936 config.status: creating tests/zfs-tests/tests/functional/mv_files/Makefile
 937 config.status: creating tests/zfs-tests/tests/functional/nestedfs/Makefile
 938 config.status: creating tests/zfs-tests/tests/functional/no_space/Makefile
 939 config.status: creating tests/zfs-tests/tests/functional/nopwrite/Makefile
 940 config.status: creating tests/zfs-tests/tests/functional/online_offline/Makefile
 941 config.status: creating tests/zfs-tests/tests/functional/pam/Makefile
 942 config.status: creating tests/zfs-tests/tests/functional/pool_checkpoint/Makefile
 943 config.status: creating tests/zfs-tests/tests/functional/pool_names/Makefile
 944 config.status: creating tests/zfs-tests/tests/functional/poolversion/Makefile
 945 config.status: creating tests/zfs-tests/tests/functional/privilege/Makefile
 946 config.status: creating tests/zfs-tests/tests/functional/procfs/Makefile
 947 config.status: creating tests/zfs-tests/tests/functional/projectquota/Makefile
 948 config.status: creating tests/zfs-tests/tests/functional/pyzfs/Makefile
 949 config.status: creating tests/zfs-tests/tests/functional/quota/Makefile
 950 config.status: creating tests/zfs-tests/tests/functional/raidz/Makefile
 951 config.status: creating tests/zfs-tests/tests/functional/redacted_send/Makefile
 952 config.status: creating tests/zfs-tests/tests/functional/redundancy/Makefile
 953 config.status: creating tests/zfs-tests/tests/functional/refquota/Makefile
 954 config.status: creating tests/zfs-tests/tests/functional/refreserv/Makefile
 955 config.status: creating tests/zfs-tests/tests/functional/removal/Makefile
 956 config.status: creating tests/zfs-tests/tests/functional/rename_dirs/Makefile
 957 config.status: creating tests/zfs-tests/tests/functional/replacement/Makefile
 958 config.status: creating tests/zfs-tests/tests/functional/reservation/Makefile
 959 config.status: creating tests/zfs-tests/tests/functional/rootpool/Makefile
 960 config.status: creating tests/zfs-tests/tests/functional/rsend/Makefile
 961 config.status: creating tests/zfs-tests/tests/functional/scrub_mirror/Makefile
 962 config.status: creating tests/zfs-tests/tests/functional/slog/Makefile
 963 config.status: creating tests/zfs-tests/tests/functional/snapshot/Makefile
 964 config.status: creating tests/zfs-tests/tests/functional/snapused/Makefile
 965 config.status: creating tests/zfs-tests/tests/functional/sparse/Makefile
 966 config.status: creating tests/zfs-tests/tests/functional/suid/Makefile
 967 config.status: creating tests/zfs-tests/tests/functional/threadsappend/Makefile
 968 config.status: creating tests/zfs-tests/tests/functional/tmpfile/Makefile
 969 config.status: creating tests/zfs-tests/tests/functional/trim/Makefile
 970 config.status: creating tests/zfs-tests/tests/functional/truncate/Makefile
 971 config.status: creating tests/zfs-tests/tests/functional/upgrade/Makefile
 972 config.status: creating tests/zfs-tests/tests/functional/user_namespace/Makefile
 973 config.status: creating tests/zfs-tests/tests/functional/userquota/Makefile
 974 config.status: creating tests/zfs-tests/tests/functional/vdev_zaps/Makefile
 975 config.status: creating tests/zfs-tests/tests/functional/write_dirs/Makefile
 976 config.status: creating tests/zfs-tests/tests/functional/xattr/Makefile
 977 config.status: creating tests/zfs-tests/tests/functional/zpool_influxdb/Makefile
 978 config.status: creating tests/zfs-tests/tests/functional/zvol/Makefile
 979 config.status: creating tests/zfs-tests/tests/functional/zvol/zvol_ENOSPC/Makefile
 980 config.status: creating tests/zfs-tests/tests/functional/zvol/zvol_cli/Makefile
 981 config.status: creating tests/zfs-tests/tests/functional/zvol/zvol_misc/Makefile
 982 config.status: creating tests/zfs-tests/tests/functional/zvol/zvol_swap/Makefile
 983 config.status: creating tests/zfs-tests/tests/perf/Makefile
 984 config.status: creating tests/zfs-tests/tests/perf/fio/Makefile
 985 config.status: creating tests/zfs-tests/tests/perf/regression/Makefile
 986 config.status: creating tests/zfs-tests/tests/perf/scripts/Makefile
 987 config.status: creating tests/zfs-tests/tests/stress/Makefile
 988 config.status: creating udev/Makefile
 989 config.status: creating udev/rules.d/Makefile
 990 config.status: creating zfs.release
 991 config.status: creating zfs_config.h
 992 config.status: executing depfiles commands
 993 config.status: executing libtool commands
 994 config.status: executing po-directories commands
 995 
 996 Building module:
 997 cleaning build area....
 998 make -j64 KERNELRELEASE=5.13.8-200.fc34.x86_64.............
 999 
1000 Running the post_build script:
1001 cleaning build area....
1002 
1003 DKMS: build completed.
1004 
1005 zavl.ko.xz:
1006 Running module version sanity check.
1007  - Original module
1008    - No original module exists within this kernel
1009  - Installation
1010    - Installing to /lib/modules/5.13.8-200.fc34.x86_64/extra/
1011 
1012 znvpair.ko.xz:
1013 Running module version sanity check.
1014  - Original module
1015    - No original module exists within this kernel
1016  - Installation
1017    - Installing to /lib/modules/5.13.8-200.fc34.x86_64/extra/
1018 
1019 zunicode.ko.xz:
1020 Running module version sanity check.
1021  - Original module
1022    - No original module exists within this kernel
1023  - Installation
1024    - Installing to /lib/modules/5.13.8-200.fc34.x86_64/extra/
1025 
1026 zcommon.ko.xz:
1027 Running module version sanity check.
1028  - Original module
1029    - No original module exists within this kernel
1030  - Installation
1031    - Installing to /lib/modules/5.13.8-200.fc34.x86_64/extra/
1032 
1033 zfs.ko.xz:
1034 Running module version sanity check.
1035  - Original module
1036    - No original module exists within this kernel
1037  - Installation
1038    - Installing to /lib/modules/5.13.8-200.fc34.x86_64/extra/
1039 
1040 icp.ko.xz:
1041 Running module version sanity check.
1042  - Original module
1043    - No original module exists within this kernel
1044  - Installation
1045    - Installing to /lib/modules/5.13.8-200.fc34.x86_64/extra/
1046 
1047 zlua.ko.xz:
1048 Running module version sanity check.
1049  - Original module
1050    - No original module exists within this kernel
1051  - Installation
1052    - Installing to /lib/modules/5.13.8-200.fc34.x86_64/extra/
1053 
1054 spl.ko.xz:
1055 Running module version sanity check.
1056  - Original module
1057    - No original module exists within this kernel
1058  - Installation
1059    - Installing to /lib/modules/5.13.8-200.fc34.x86_64/extra/
1060 
1061 zzstd.ko.xz:
1062 Running module version sanity check.
1063  - Original module
1064    - No original module exists within this kernel
1065  - Installation
1066    - Installing to /lib/modules/5.13.8-200.fc34.x86_64/extra/
1067 
1068 depmod......
1069 Warning: The unit file, source configuration file or drop-ins of systemd-modules-load.service changed on disk. Run 'systemctl daemon-reload' to reload units.
1070 
1071 DKMS: install completed.
1072  Done. 
1073 dkms: running auto installation service for kernel 5.13.8-200.fc34.x86_64
1074  Done. 

Here is an example of one of the deployments:

[root@vault ~]# zfs create app/hedgedoc

[root@vault ~]# zfs set quota=5T app/hedgedoc

[root@vault ~]# zfs list | grep hedgedoc

app/hedgedoc               188M  5.00T      188M  /zfs/app/hedgedoc

[root@vault ~]# useradd -g 602 -u 602 -d /zfs/app/hedgedoc/ -s /sbin/nologin hedgedoc

[root@vault ~]# usermod --add-subuids 60200000-60265535 --add-subgids 60200000-60265535 hedgedoc

[root@vault ~]# loginctl enable-linger hedgedoc

[root@vault ~]# runuser -l -s /bin/bash hedgedoc

[hedgedoc@vault ~]$ podman info

host:
  arch: amd64
  buildahVersion: 1.21.3
  cgroupControllers: []
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.0.29-2.fc34.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.29, commit: '
  cpus: 64
  distribution:
    distribution: fedora
    version: "34"
  eventLogger: journald
  hostname: vault
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 602
      size: 1
    - container_id: 1
      host_id: 60200000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 602
      size: 1
    - container_id: 1
      host_id: 60200000
      size: 65536
  kernel: 5.13.8-200.fc34.x86_64
  linkmode: dynamic
  memFree: 264640901120
  memTotal: 270109155328
  ociRuntime:
    name: crun
    package: crun-0.20.1-1.fc34.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 0.20.1
      commit: 0d42f1109fd73548f44b01b3e84d04a279e99d2e
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    path: /run/user/602/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.1.9-1.fc34.x86_64
    version: |-
      slirp4netns version 1.1.8+dev
      commit: 6dc0186e020232ae1a6fcc1f7afbc3ea02fd3876
      libslirp: 4.4.0
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.0
  swapFree: 8589930496
  swapTotal: 8589930496
  uptime: 26m 57.36s
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
  - quay.io
store:
  configFile: /zfs/app/hedgedoc/.config/containers/storage.conf
  containerStore:
    number: 2
    paused: 0
    running: 1
    stopped: 1
  graphDriverName: overlay
  graphOptions:
    overlay.mount_program:
      Executable: /usr/bin/fuse-overlayfs
      Package: fuse-overlayfs-1.7.0-1.fc34.x86_64
      Version: |-
        fusermount3 version: 3.10.4
        fuse-overlayfs: version 1.7
        FUSE library version 3.10.4
        using FUSE kernel interface version 7.31
  graphRoot: /zfs/app/hedgedoc/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: zfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 2
  runRoot: /run/user/602/containers
  volumePath: /zfs/app/hedgedoc/.local/share/containers/storage/volumes
version:
  APIVersion: 3.2.3
  Built: 1627933161
  BuiltTime: Mon Aug  2 15:39:21 2021
  GitCommit: ""
  GoVersion: go1.16.6
  OsArch: linux/amd64
  Version: 3.2.3

[hedgedoc@vault ~]$ podman pod create --name hedgedoc --publish 127.0.0.1:8388:3000

063264f3461bbd653ca3a3669441bafd2135514cf0d457f16542a5a386633a89

[hedgedoc@vault ~]$ podman pod ps

POD ID        NAME        STATUS      CREATED         INFRA ID      # OF CONTAINERS
063264f3461b  hedgedoc    Created     13 seconds ago  e393e34dd5d5  1

[hedgedoc@vault ~]$ mkdir ~/database

Assume that I have created a proper file at ~/.database.env with envars in it.

[hedgedoc@vault ~]$ podman run --pod=hedgedoc --name=database --label "io.containers.autoupdate=image" --env-file /zfs/app/hedgedoc/database.env --volume /zfs/app/hedgedoc/database/:/var/lib/postgresql/data:Z docker.io/postgres:13

Error: OCI runtime error: mkdir `data`: Operation not supported

From what I gather this is:

  • nothing to do with selinux (turn it to permissive and nothing changes)
  • nothing to do with permissions, can set the ~/database directory to 777
  • nothing to do with prior podman stuff in the hedgedoc users home directory (I removed this user and the home directory to start fresh)

I immediately downgraded to podman 3.1.0 to see if that was the case as well, but that behaves the same way.

I'm actually super stumped on this one. Maybe it's ZFS?

@bcstinch
Copy link

I can also reproduce this issue on Fedora 34 and podman 3.2.3. However, I am not seeing this issue if I run podman as root.

@rhatdan
Copy link
Member

rhatdan commented Aug 12, 2021

Could this be a fuse-overlayfs issue?

@andrewgdunn
Copy link
Author

@rhatdan would there be something I could do to test that?

I can confirm, this seems to only manifest when I have a user home directory, and the associated bind mounts, on a ZFS dataset. I moved all my applications temporarily to mdadm+xfs and everything comes up as expected.

This is a bit of a pickle. I realize that ZFS is out of tree and all that, also not golden path for RH or Fedora. Wondering if I should register an issue upstream with that project or if this is within the podman ecosystem?

@rhatdan
Copy link
Member

rhatdan commented Aug 16, 2021

@giuseppe @mheon thoughts?

@andrewgdunn
Copy link
Author

To report back in:

  • --volume appears to not function when the filesystem is ZFS
    • You get an OCI runtime error for whatever filesystem operation you're trying to perform (e.g. Error: OCI runtime error: mkdir)
  • --mount works just fine, as expected, read/write without issue

I'm not sure if this project is the right upstream to be reporting in, would there be a better place to report (or link) and attempt to characterize this issue further? I've got a system that I can rapidly test things on if there are any suggestions.

@mheon
Copy link
Member

mheon commented Aug 16, 2021

Alright, that doesn't make much sense... Under the hood, those should both be kernel bind-mounts, with almost identical options.

Can you include examples of working --mount flags vs not-working --volume flags? And, ideally, a podman inspect of container with a working mount vs a not-working volume - that should include detailed options for the bind-mounts.

@c-goes
Copy link

c-goes commented Aug 23, 2021

To report back in:

* `--volume` appears to not function when the filesystem is ZFS

I can't confirm this. I'm using podman with fuse-overlayfs on Ubuntu and my home directory is ZFS. --volume works perfectly.

@maxaudron
Copy link

I'm not sure if this is exactly related, but this seems like a good place to put this for now, spent most of my day trying to figure this out after updating my system. I started to get this error with some containers but not all:

Error: error committing container for step {Env:[PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin] Command:run Args:[adduser -D test] Flags:[] Attrs:map[] Message:RUN adduser -D test Original:RUN adduser -D test}: error copying layers and metadata for container "087335ed9794ebe416561e11e8bd8d81a5f3f5d11f81a9060f1c75eb84ef7163": initializing source containers-storage:alpine-working-container: error extracting layer "42b399d8ccbb13175d8fcd4da261d78d73664d7c73c124058b108753c88b7708": open /home/audron/.local/share/containers/storage/overlay/42b399d8ccbb13175d8fcd4da261d78d73664d7c73c124058b108753c88b7708/merged/home/test: operation not supported

and stuff like

{"msg":"exec container process `/docker-entrypoint.sh`: Operation not supported","level":"error","time":"2021-09-14T16:16:01.000973906Z"}

when trying to run containers.

After trying around for a bit, I found out that specifically the make home directory part of adduser -D test in an alpine container was causing this to happen, doing adduser -D -H test worked fine.

Tried my way through podman versions back to 2.2.0, and it was the same everywhere. The only thing that got updated recently was fuse-overlayfs, from 1.5 to 1.7.1. I was able to narrow the cause down to this commit: containers/fuse-overlayfs@dd7e124, which enabled POSIX ACLs.

But as it turns out my zfs pools on gentoo had acltype=off set, after setting it to acltype=posixacl and xattr=sa (mainly for performance reasons) and deleting my ~/.local/share/containers it worked again.

Oddly enough, I'm also encountering this on my server where I run buildah in cri-o containers for CI, where the zfs pools already have this property set. Here the docker.io/node:alpine images is causing it.

@giuseppe
Copy link
Member

opened a PR for fuse-overlayfs so it is possible to disable ACLs: containers/fuse-overlayfs#323

@andrewgdunn
Copy link
Author

@maxaudron

But as it turns out my zfs pools on gentoo had acltype=off set, after setting it to acltype=posixacl and xattr=sa (mainly for performance reasons) and deleting my ~/.local/share/containers it worked again.

Trying to pull this apart:

  • acltype=off: works?
  • acltype=posixacl or xattr=sa: no works?

@andrewgdunn
Copy link
Author

Actually re-reading, it sounds like acltype=off was the problem, but acltype=posixacl works (until @giuseppe flag for disabling ACLs is in a release).

@andrewgdunn
Copy link
Author

Was able to reproduce with acltype=posixacl and xattr=sa. I don't know how critical xattr=sa was, but it's something that I've turned on by convention on pools for a long time.

Thanks @maxaudron for chiming in, this has been making me sad for a while.

@giuseppe out of the box podman and fuse-overlayfs will be wanting ACL(s) right? So if this were to be a tombstone for people to find they could do the following:

  • turn of ACL(s) via the pull request you reference above when invoking podman
  • set acltype=posixacl on their ZFS dataset.

@andrewgdunn andrewgdunn changed the title Bind mounts permissions issues on ZFS? ZFS+acltype+fuse-overlayfs, Operation Not Supported Oct 7, 2021
@giuseppe
Copy link
Member

@storrgie yes, by default ACLs are enabled.

Is there anything more that should happen in Podman/fuse-overlayfs or can we close this issue now?

@andrewgdunn
Copy link
Author

We can close it, just wanted to have it be a completely understood phenomena. Thanks!

@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 21, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 21, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.
Projects
None yet
Development

No branches or pull requests

7 participants