Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update to 4.0.0 release #14

Draft
wants to merge 11 commits into
base: master
Choose a base branch
from
Draft

update to 4.0.0 release #14

wants to merge 11 commits into from

Conversation

daltonbohning
Copy link

@daltonbohning daltonbohning commented Jan 12, 2024

  • update to 4.0.0 release
  • update packaging

Signed-off-by: Dalton Bohning <[email protected]>
@daltonbohning daltonbohning self-assigned this Jan 12, 2024
@daltonbohning daltonbohning requested a review from a team as a code owner January 12, 2024 18:57
@daltonbohning daltonbohning marked this pull request as draft January 12, 2024 18:58
Signed-off-by: Dalton Bohning <[email protected]>
Signed-off-by: Dalton Bohning <[email protected]>
Signed-off-by: Dalton Bohning <[email protected]>
Signed-off-by: Dalton Bohning <[email protected]>
Signed-off-by: Dalton Bohning <[email protected]>
URL: https://github.com/hpc/%{name}/
Source0: https://github.com/hpc/%{name}/releases/download/%{version}/%{name}-%{version}.tar.gz
%if "%{?commit}" != ""
Patch1: %{version}..%{commit}.patch
%endif
Patch3: daos-configure.patch
# patch configure.ac
Patch3: https://github.com/hpc/ior/commit/38064419cbe959cb538695e51b2bc2a91d6971f7.patch
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also just update to 4.0.0 + patch tip of main. Which is what the spec does currently. I.e. 3.3.0..d3574d536643475269d37211e283b49ebd6732d7.patch

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could, at the risk of introducing post-4.0.0 landings that could be troublesome. I think what we are doing here is better and more idiomatic of software packaging.


Summary: IOR-HPC

License: GPL
License: GPL-2.0-only
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To resolve invalid-license lint

Comment on lines +103 to +105
%find_lang mdtest --with-man
%files -f files.mpich
%files -f mdtest.lang
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolves file-not-in-%lang

@daltonbohning
Copy link
Author

@brianjmurrell Any advice on how to fix this lint warning for leap 15.5?
https://build.hpdd.intel.com/blue/rest/organizations/jenkins/pipelines/daos-stack/pipelines/ior/branches/PR-14/runs/8/nodes/206/log/?start=0

RPM Lint found warnings:
ior.x86_64: W: position-independent-executable-suggested /usr/bin/ior
ior.x86_64: W: position-independent-executable-suggested /usr/bin/md-workbench
ior.x86_64: W: position-independent-executable-suggested /usr/bin/mdtest

I thought this was handled by

ior/ior.spec

Lines 61 to 62 in ecea31d

export CFLAGS="$RPM_OPT_FLAGS -fno-strict-aliasing -fPIC"
export CXXFLAGS="$RPM_OPT_FLAGS -fno-strict-aliasing -fPIC"

@brianjmurrell
Copy link
Contributor

I thought this was handled by

ior/ior.spec

Lines 61 to 62 in ecea31d

export CFLAGS="$RPM_OPT_FLAGS -fno-strict-aliasing -fPIC"
export CXXFLAGS="$RPM_OPT_FLAGS -fno-strict-aliasing -fPIC"

I would think so too. As long at the ior build system was not overriding or ignoring those in some way. Does the build log show that they are being used? You might have to add a V=1 to the make arguments to get verbose command lines.

They in fact shouldn't even be necessary as hardened_build is default in EL > 7. Ahhh. But maybe necessary still for Leap. If so, we ought to guard that so that it doesn't override the goodness that RHEL bakes into their RPM build macros.

@daltonbohning
Copy link
Author

Does the build log show that they are being used?

For EL9, I see -fPIC in the build commands, but for Leap I do not. I'll try with -V.

Signed-off-by: Dalton Bohning <[email protected]>
@daltonbohning
Copy link
Author

daltonbohning commented Jan 16, 2024

It looks like it's using it?
https://build.hpdd.intel.com/job/daos-stack/job/ior/job/PR-14/9/artifact/artifacts/leap15/build.log/*view*/

Making all in .
mpicc -DHAVE_CONFIG_H -I.      -Icheck/include -Icheck/include -I/usr/include  -O2 -g -m64 -fmessage-length=0 -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -fno-strict-aliasing -fPIC -c -o md_workbench-md-workbench-main.o `test -f 'md-workbench-main.c' || echo './'`md-workbench-main.c
...

Specifically, for the mdtest executable

mpicc  -O2 -g -m64 -fmessage-length=0 -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -fno-strict-aliasing -fPIC     -Lcheck/lib64 -Wl,--enable-new-dtags -Wl,-rpath=check/lib64 -Lcheck/lib64 -Wl,--enable-new-dtags -Wl,-rpath=check/lib64 -L/usr/lib64 -Wl,--enable-new-dtags -Wl,-rpath=/usr/lib64 -o mdtest mdtest-mdtest-main.o mdtest-aiori.o mdtest-aiori-DUMMY.o   mdtest-aiori-HDF5.o  mdtest-aiori-MPIIO.o  mdtest-aiori-MMAP.o

But lint complains

RPM Lint found warnings:
ior.x86_64: W: position-independent-executable-suggested /usr/bin/ior
ior.x86_64: W: position-independent-executable-suggested /usr/bin/md-workbench
ior.x86_64: W: position-independent-executable-suggested /usr/bin/mdtest

@daltonbohning
Copy link
Author

Ah. Maybe it's -fPIC (position-independent-code) vs -fPIE (position-independent-executable)

@daltonbohning
Copy link
Author

It seems -fPIC should be used when creating the .so, and -fPIE when creating the executables

@daltonbohning
Copy link
Author

Looking at the EL9 log for the mdtest executable, -fPIC is used but not -fPIE. So I'm confused
https://build.hpdd.intel.com/job/daos-stack/job/ior/job/PR-14/9/artifact/artifacts/el9/build.log

mpicc  -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64 -march=x86-64-v2 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fno-strict-aliasing -fPIC     -Wl,-z,relro -Wl,--as-needed  -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Lcheck/lib64 -Wl,--enable-new-dtags -Wl,-rpath=check/lib64 -Lcheck/lib64 -Wl,--enable-new-dtags -Wl,-rpath=check/lib64 -L/usr/lib64 -Wl,--enable-new-dtags -Wl,-rpath=/usr/lib64 -o mdtest mdtest-mdtest-main.o mdtest-aiori.o mdtest-aiori-DUMMY.o   mdtest-aiori-HDF5.o  mdtest-aiori-MPIIO.o  mdtest-aiori-MMAP.o mdtest-aiori-POSIX.o     mdtest-aiori-DFS.o     libaiori.a    -lhdf5 -lz       -lgurt -ldaos_common -ldaos -ldfs -luuid      -ldfs -ldaos -luuid -lgurt -lhdf5 -lm 

Comment on lines +61 to +62
export CFLAGS="$RPM_OPT_FLAGS -fno-strict-aliasing -fPIC -fPIE"
export CXXFLAGS="$RPM_OPT_FLAGS -fno-strict-aliasing -fPIC -fPIE"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since EL >= 8 defaults to hardening their builds, we should gate these two lines on being SUSE only:

Suggested change
export CFLAGS="$RPM_OPT_FLAGS -fno-strict-aliasing -fPIC -fPIE"
export CXXFLAGS="$RPM_OPT_FLAGS -fno-strict-aliasing -fPIC -fPIE"
%if (0%{?suse_version} >= 1500)
export CFLAGS="$RPM_OPT_FLAGS -fno-strict-aliasing -fPIC -fPIE"
export CXXFLAGS="$RPM_OPT_FLAGS -fno-strict-aliasing -fPIC -fPIE"
%endif

so that we don't interfere with EL doing the right thing on it's own.

%endif

%changelog
* Fri Jan 12 2024 Dalton A. Bohning <[email protected]> - 4.0.0-1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's do some datestamp updating here at some point since this is now 10 days ago.

@brianjmurrell
Copy link
Contributor

These:

ior.x86_64: W: position-independent-executable-suggested /usr/bin/ior
ior.x86_64: W: position-independent-executable-suggested /usr/bin/md-workbench
ior.x86_64: W: position-independent-executable-suggested /usr/bin/mdtest

are really odd given that we can see that the link (and all of the object compilations) are using the hardening flags:

mpicc  -O2 -g -m64 -fmessage-length=0 -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -fno-strict-aliasing -fPIC -fPIE     -Lcheck/lib64 -Wl,--enable-new-dtags -Wl,-rpath=check/lib64 -Lcheck/lib64 -Wl,--enable-new-dtags -Wl,-rpath=check/lib64 -L/usr/lib64 -Wl,--enable-new-dtags -Wl,-rpath=/usr/lib64 -o mdtest mdtest-mdtest-main.o mdtest-aiori.o mdtest-aiori-DUMMY.o   mdtest-aiori-HDF5.o  mdtest-aiori-MPIIO.o  mdtest-aiori-MMAP.o mdtest-aiori-POSIX.o     mdtest-aiori-DFS.o     libaiori.a    -lhdf5 -lz       -lgurt -ldaos_common -ldaos -ldfs -luuid      -ldfs -ldaos -luuid -lgurt -lhdf5 -lm 
mpicc  -O2 -g -m64 -fmessage-length=0 -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -fno-strict-aliasing -fPIC -fPIE     -Lcheck/lib64 -Wl,--enable-new-dtags -Wl,-rpath=check/lib64 -Lcheck/lib64 -Wl,--enable-new-dtags -Wl,-rpath=check/lib64 -L/usr/lib64 -Wl,--enable-new-dtags -Wl,-rpath=/usr/lib64 -o ior ior-ior-main.o ior-aiori.o ior-aiori-DUMMY.o   ior-aiori-HDF5.o  ior-aiori-MPIIO.o  ior-aiori-MMAP.o ior-aiori-POSIX.o     ior-aiori-DFS.o     libaiori.a    -lhdf5 -lz       -lgurt -ldaos_common -ldaos -ldfs -luuid      -ldfs -ldaos -luuid -lgurt -lhdf5 -lm 
mpicc  -O2 -g -m64 -fmessage-length=0 -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -fno-strict-aliasing -fPIC -fPIE     -Lcheck/lib64 -Wl,--enable-new-dtags -Wl,-rpath=check/lib64 -Lcheck/lib64 -Wl,--enable-new-dtags -Wl,-rpath=check/lib64 -L/usr/lib64 -Wl,--enable-new-dtags -Wl,-rpath=/usr/lib64 -o md-workbench md_workbench-md-workbench-main.o md_workbench-aiori.o md_workbench-aiori-DUMMY.o   md_workbench-aiori-HDF5.o  md_workbench-aiori-MPIIO.o  md_workbench-aiori-MMAP.o md_workbench-aiori-POSIX.o     md_workbench-aiori-DFS.o     libaiori.a    -lhdf5 -lz       -lgurt -ldaos_common -ldaos -ldfs -luuid      -ldfs -ldaos -luuid -lgurt -lhdf5 -lm 

This is the link command from EL8:

mpicc  -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fno-strict-aliasing -fPIC -fPIE     -Wl,-z,relro  -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Lcheck/lib64 -Wl,--enable-new-dtags -Wl,-rpath=check/lib64 -Lcheck/lib64 -Wl,--enable-new-dtags -Wl,-rpath=check/lib64 -L/usr/lib64 -Wl,--enable-new-dtags -Wl,-rpath=/usr/lib64 -o ior ior-ior-main.o ior-aiori.o ior-aiori-DUMMY.o   ior-aiori-HDF5.o  ior-aiori-MPIIO.o  ior-aiori-MMAP.o ior-aiori-POSIX.o     ior-aiori-DFS.o     libaiori.a    -lhdf5 -lz       -lgurt -ldaos_common -ldaos -ldfs -luuid      -ldfs -ldaos -luuid -lgurt -lhdf5 -lm 
mpicc  -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fno-strict-aliasing -fPIC -fPIE     -Wl,-z,relro  -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Lcheck/lib64 -Wl,--enable-new-dtags -Wl,-rpath=check/lib64 -Lcheck/lib64 -Wl,--enable-new-dtags -Wl,-rpath=check/lib64 -L/usr/lib64 -Wl,--enable-new-dtags -Wl,-rpath=/usr/lib64 -o mdtest mdtest-mdtest-main.o mdtest-aiori.o mdtest-aiori-DUMMY.o   mdtest-aiori-HDF5.o  mdtest-aiori-MPIIO.o  mdtest-aiori-MMAP.o mdtest-aiori-POSIX.o     mdtest-aiori-DFS.o     libaiori.a    -lhdf5 -lz       -lgurt -ldaos_common -ldaos -ldfs -luuid      -ldfs -ldaos -luuid -lgurt -lhdf5 -lm 
mpicc  -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fno-strict-aliasing -fPIC -fPIE     -Wl,-z,relro  -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Lcheck/lib64 -Wl,--enable-new-dtags -Wl,-rpath=check/lib64 -Lcheck/lib64 -Wl,--enable-new-dtags -Wl,-rpath=check/lib64 -L/usr/lib64 -Wl,--enable-new-dtags -Wl,-rpath=/usr/lib64 -o md-workbench md_workbench-md-workbench-main.o md_workbench-aiori.o md_workbench-aiori-DUMMY.o   md_workbench-aiori-HDF5.o  md_workbench-aiori-MPIIO.o  md_workbench-aiori-MMAP.o md_workbench-aiori-POSIX.o     md_workbench-aiori-DFS.o     libaiori.a    -lhdf5 -lz       -lgurt -ldaos_common -ldaos -ldfs -luuid      -ldfs -ldaos -luuid -lgurt -lhdf5 -lm 

RH puts all of their hardening in gcc with this option -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 which is simply:

*cc1_options:
+ %{!r:%{!fpie:%{!fPIE:%{!fpic:%{!fPIC:%{!fno-pic:-fPIE}}}}}}

which we can is some flavour of -fPIE.

Signed-off-by: Dalton Bohning <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants