-dmks RPM packages require ksh in 0.7.0

[Lalufu]

System information

Type	Version/Name
Distribution Name	Fedora
Distribution Version	25
Linux Kernel	4.11.5
Architecture	x86_64
ZFS Version	0.7.0
SPL Version	0.7.0

Describe the problem you're observing

Installing the 0.7.0 dkms RPMs requires /bin/ksh, the 0.6.x versions did not. From a cursory glance at the .spec file it's not immediately obvious to me where this dependency comes from, but it might be that rpmbuild picks up the test case files. This is mostly a cosmetic issue, tbh.

[loli10K]

but it might be that rpmbuild picks up the test case files.

You are spot on here:

[root@centos7 ~]# rpm -qlp ./zfs-dkms-0.6.5.8-1.el7_3.centos.noarch.rpm | grep tests
/usr/src/zfs-0.6.5.8/lib/libspl/include/sys/feature_tests.h
[root@centos7 ~]# rpm -qlp ./zfs-dkms-0.7.0-1.el7_3.noarch.rpm | grep -c tests
1705
[root@centos7 ~]# rpm -qvp --requires ./zfs-dkms-0.7.0-1.el7_3.noarch.rpm
auto: /bin/bash
auto: /bin/ksh
auto: /bin/sed
post,interp: /bin/sh
preun,interp: /bin/sh
auto: /bin/sh
auto: /usr/bin/python
manual: dkms >= 2.2.0.3-20
manual: gcc
manual: kernel-devel
manual: make
manual: perl
rpmlib: rpmlib(CompressedFileNames) <= 3.0.4-1
rpmlib: rpmlib(FileDigests) <= 4.6.0-1
rpmlib: rpmlib(PayloadFilesHavePrefix) <= 4.0-1
manual: spl-dkms = 0.7.0
rpmlib: rpmlib(PayloadIsXz) <= 5.2-1
[root@centos7 ~]# 

/bin/ksh is "auto"-added as a dependency when we scan the source directory which includes the ZTS.

If desired we could exclude the test suite from the dkms package with something like this:

diff --git a/rpm/generic/zfs-dkms.spec.in b/rpm/generic/zfs-dkms.spec.in
index e432bc4..d438a5f 100644
--- a/rpm/generic/zfs-dkms.spec.in
+++ b/rpm/generic/zfs-dkms.spec.in
@@ -45,6 +45,7 @@ fi
 
 %files
 %defattr(-,root,root)
+%exclude /usr/src/%{module}-%{version}/tests
 /usr/src/%{module}-%{version}
 
 %post

[behlendorf]

We should really exclude anything which isn't a kmod. The zfs-dkms package is only intended to provide the kernel modules, nothing for user space.

[kkretschmer]

This problem will become more annoying with Fedora 27 (currently still in beta). There, the ksh package does not provide /bin/ksh any more (it does provide /usr/bin/ksh), so dnf refuses to install the zfs-dkms and zfs packages and I had do force installation with rpm -i --nodeps. The %exclude approch in a previous comment did not work for me, zfs-dkms still has the dependency auto: /bin/ksh. The %exclude approach worked for me on Fedora 27, zfs-0.7.3.

[behlendorf]

@kkretschmer does disabling automatic dependency generation avoid the issue. This should be fine as a workaround for the zfs-dkms package.

diff --git a/rpm/generic/zfs-dkms.spec.in b/rpm/generic/zfs-dkms.spec.in
index f6de492..73a774b 100644
--- a/rpm/generic/zfs-dkms.spec.in
+++ b/rpm/generic/zfs-dkms.spec.in
@@ -27,6 +27,7 @@ Requires:       gcc, make, perl
 Requires:       kernel-devel
 %endif
 Provides:       %{module}-kmod = %{version}
+AutoReqProv:    no
 
 %description
 This package contains the dkms ZFS kernel modules.

[kkretschmer]

Yes, it does. I applied this change to zfs-0.7.3, rebuilt rpms and zfs-dkms now installs without dependency problems. Output of rpm -q --requires zfs-dkms-0.7.3-1.fc27.noarch.rpm:

diff --git a/requires.orig b/requires
index df660a7..6f76383 100644
--- a/requires.orig
+++ b/requires
@@ -1,21 +1,10 @@
-/bin/bash
-/bin/ksh
-/bin/sed
 /bin/sh
 /bin/sh
-/bin/sh
-/usr/bin/env
-/usr/bin/python
 dkms >= 2.2.0.3
 gcc
 kernel-devel
 make
 perl
-perl(:VERSION) >= 5.0.0
-perl(Getopt::Std)
-perl(IO::File)
-perl(strict)
-perl(warnings)
 rpmlib(CompressedFileNames) <= 3.0.4-1
 rpmlib(FileDigests) <= 4.6.0-1
 rpmlib(PartialHardlinkSets) <= 4.0.4-1

[loli10K]

Excluding /usr/src/%{module}-%{version}/tests, at least on my CentOS7 box, produces the following zfs-dkms package (without any ksh dependency):

diff --git a/rpm/generic/zfs-dkms.spec.in b/rpm/generic/zfs-dkms.spec.in
index f6de492..5deba15 100644
--- a/rpm/generic/zfs-dkms.spec.in
+++ b/rpm/generic/zfs-dkms.spec.in
@@ -51,6 +51,7 @@ fi
 
 %files
 %defattr(-,root,root)
+%exclude /usr/src/%{module}-%{version}/tests
 /usr/src/%{module}-%{version}
 
 %post
[root@centos zfs]# rpm -qvp --requires ./zfs-dkms-0.7.0-163_gdf1f129.el7.centos.noarch.rpm 
auto: /bin/bash
auto: /bin/sed
post,interp: /bin/sh
preun,interp: /bin/sh
auto: /bin/sh
auto: /usr/bin/python
manual: dkms >= 2.2.0.3
manual: gcc
manual: kernel-devel
manual: make
manual: perl
rpmlib: rpmlib(CompressedFileNames) <= 3.0.4-1
rpmlib: rpmlib(FileDigests) <= 4.6.0-1
rpmlib: rpmlib(PartialHardlinkSets) <= 4.0.4-1
rpmlib: rpmlib(PayloadFilesHavePrefix) <= 4.0-1
manual: spl-dkms = 0.7.0
rpmlib: rpmlib(PayloadIsXz) <= 5.2-1
[root@centos zfs]# rpm -qvp --requires ./zfs-dkms-0.7.0-163_gdf1f129.el7.centos.noarch.rpm | grep ksh
[root@centos zfs]# 

EDIT: Just tested Fedora27

[root@fedora27 zfs]# git diff
diff --git a/rpm/generic/zfs-dkms.spec.in b/rpm/generic/zfs-dkms.spec.in
index f6de49297..5deba1551 100644
--- a/rpm/generic/zfs-dkms.spec.in
+++ b/rpm/generic/zfs-dkms.spec.in
@@ -51,6 +51,7 @@ fi
 
 %files
 %defattr(-,root,root)
+%exclude /usr/src/%{module}-%{version}/tests
 /usr/src/%{module}-%{version}
 
 %post
[root@fedora27 zfs]# rpm -qvp --requires ./zfs-dkms-0.7.0-165_gcd1813d36.fc27.noarch.rpm 
auto: /bin/bash
auto: /bin/sed
post,interp: /bin/sh
preun,interp: /bin/sh
auto: /bin/sh
auto: /usr/bin/env
auto: /usr/bin/python
manual: dkms >= 2.2.0.3
manual: gcc
manual: kernel-devel
manual: make
manual: perl
rpmlib: rpmlib(CompressedFileNames) <= 3.0.4-1
rpmlib: rpmlib(FileDigests) <= 4.6.0-1
rpmlib: rpmlib(PartialHardlinkSets) <= 4.0.4-1
rpmlib: rpmlib(PayloadFilesHavePrefix) <= 4.0-1
rpmlib: rpmlib(PayloadIsXz) <= 5.2-1
manual: spl-dkms = 0.7.0
[root@fedora27 zfs]# rpm -qvp --requires ./zfs-dkms-0.7.0-165_gcd1813d36.fc27.noarch.rpm | grep ksh
[root@fedora27 zfs]# 

[kkretschmer]

Sorry, I probably made a mistake in my test of the %exclude statement. Starting from a fresh clone of the zfs-0.7.3 tag, it works.
Output of git diff zfs-0.7.3:

diff --git a/rpm/generic/zfs-dkms.spec.in b/rpm/generic/zfs-dkms.spec.in
index f6de49297..5deba1551 100644
--- a/rpm/generic/zfs-dkms.spec.in
+++ b/rpm/generic/zfs-dkms.spec.in
@@ -51,6 +51,7 @@ fi
 
 %files
 %defattr(-,root,root)
+%exclude /usr/src/%{module}-%{version}/tests
 /usr/src/%{module}-%{version}
 
 %post

Diffed output of rpm -qvp --requires ./zfs-dkms-0.7.3-1.fc27.noarch.rpm:

diff --git a/requires.orig b/requires.exclude
index ba43c66..093367f 100644
--- a/requires.orig
+++ b/requires.exclude
@@ -1,5 +1,4 @@
 auto: /bin/bash
-auto: /bin/ksh
 auto: /bin/sed
 post,interp: /bin/sh
 preun,interp: /bin/sh

[behlendorf]

So the trade off with using %exclude is that those files will be removed from the package entirely. That means you'll no longer be able to run a normal ./configure in the dkms directory since the build system expects certain paths to exist and they no longer will. You'll only be able to run ./configure --with-config=kernel. This isn't necessarily a problem because that's the only way the dkms scripts will build things but it's something to keep in mind. And if we do decide that's the way to go we might as well exclude a few other unused directories.

Alternately, disabling the automatic dependency generation and explicitly listing the dependencies allows for the entire package to be buildable from the dkms directory. Now how useful that is I'd say is debatable.

@loli10K @kkretschmer do you have a preference?

[kkretschmer]

I personally have only ever built packages from a git working tree and used the files installed by the zfs-dkms package only for the dkms tool. The two approaches would not affect this. I’m not feeling very confident that I fully understand all the tradeoffs involved, but the %exclude approach seems more elegant to me. I think that someone who wants to run the tests will probably also have a clone of the repository.

[loli10K]

I'm leaning toward the AutoReqProv: no fix, mostly because i like how it feels more like a "whitelist" approach (instead of listing every "blacklisted" directory) and also because i just tried to install the rpm package generated with the %exclude option and that doesn't seem to work properly:

[root@centos zfs]# rpm -qvp --requires zfs-dkms-0.7.0-163_gdf1f129.el7.centos.noarch.rpm
auto: /bin/bash
auto: /bin/sed
post,interp: /bin/sh
preun,interp: /bin/sh
auto: /bin/sh
auto: /usr/bin/python
manual: dkms >= 2.2.0.3
manual: gcc
manual: kernel-devel
manual: make
manual: perl
rpmlib: rpmlib(CompressedFileNames) <= 3.0.4-1
rpmlib: rpmlib(FileDigests) <= 4.6.0-1
rpmlib: rpmlib(PartialHardlinkSets) <= 4.0.4-1
rpmlib: rpmlib(PayloadFilesHavePrefix) <= 4.0-1
manual: spl-dkms = 0.7.0
rpmlib: rpmlib(PayloadIsXz) <= 5.2-1
[root@centos zfs]# yum localinstall -y zfs-dkms-0.7.0-163_gdf1f129.el7.centos.noarch.rpm
Loaded plugins: fastestmirror
Examining zfs-dkms-0.7.0-163_gdf1f129.el7.centos.noarch.rpm: zfs-dkms-0.7.0-163_gdf1f129.el7.centos.noarch
Marking zfs-dkms-0.7.0-163_gdf1f129.el7.centos.noarch.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package zfs-dkms.noarch 0:0.7.0-163_gdf1f129.el7.centos will be installed
--> Finished Dependency Resolution

Dependencies Resolved

[...]

Install  1 Package

Total size: 15 M
Installed size: 15 M
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Warning: RPMDB altered outside of yum.
  Installing : zfs-dkms-0.7.0-163_gdf1f129.el7.centos.noarch                                                                                                                                                   1/1 
Loading new zfs-0.7.0 DKMS files...
Building for 3.10.0-693.5.2.el7.x86_64
Building initial module for 3.10.0-693.5.2.el7.x86_64
config.status: error: cannot find input file: `tests/Makefile.in'
Error! Bad return status for module build on kernel: 3.10.0-693.5.2.el7.x86_64 (x86_64)
Consult /var/lib/dkms/zfs/0.7.0/build/make.log for more information.
warning: %post(zfs-dkms-0.7.0-163_gdf1f129.el7.centos.noarch) scriptlet failed, exit status 10
Non-fatal POSTIN scriptlet failure in rpm package zfs-dkms-0.7.0-163_gdf1f129.el7.centos.noarch
  Verifying  : zfs-dkms-0.7.0-163_gdf1f129.el7.centos.noarch                                                                                                                                                   1/1 

Installed:
  zfs-dkms.noarch 0:0.7.0-163_gdf1f129.el7.centos                                                                                                                                                                  

Complete!

[behlendorf]

Alright, let's go with the AutoReqProv: no solution which is my preference too. On the kernel side our dependencies are minimal and I'd be surprised if they weren't all, or almost all, already satisfied by installing the dkms package itself. @loli10K do you want to open a PR for this or shall I?

[loli10K]

@behlendorf please do, it's past midnight here and i was going to take a nap; i don't think i'll be able to stay awake for much longer, sorry.

[behlendorf]

Done, #6835.