Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set SOURCE_DATE_EPOCH (2nd try) #62

Merged
merged 1 commit into from
Feb 15, 2024

Conversation

JanZerebecki
Copy link
Member

@JanZerebecki JanZerebecki commented Feb 1, 2024

replacing rpm its %source_date_epoch_from_changelog . This instead also
increments the date by the build counter obtained from OBS. When
available it uses the mtime that OBS its scmsync obtains from the git
HEAD commit date instead of from the changelog.

It exports the old date in SOURCE_DATE_EPOCH and the new incremented
date in SOURCE_DATE_EPOCH_MTIME. rpm will use
SOURCE_DATE_EPOCH_MTIME for for file mtimes and the old date for
everything else.

All this can be disabled by setting the rpm macro
%disable_obs_set_source_date_epoch to Y .

This uses new variables in /.buildenv that were implemented in
openSUSE/obs-build@5d1da85
.

See
https://reproducible-builds.org/docs/source-date-epoch/#interaction-of-source_date_epoch-with-automatic-rebuilds
for more general discussion.

Increasing the date by the build counter is necessary, so that that even
for rebuilds with unchanged source but updated build depends the date is
always increased, such that no build with a different input/output has
the same date.

Not increasing the date breaks rsync without --checksum or anything else
that relies on modification time stamps of files to detect changes.

When %clamp_mtime_to_source_date_epoch and
%source_date_epoch_from_changelog are enabled and a rebuild is done with
different build dependencies a file may have different content but with
the same mtime. When two releases of this rpm are extracted after each
other at the same location and the extracted files are synced with rsync
without --checksum, then the resulting copy will contain a mix of files
from the old and new build. This can happen on installed systems due to
updates when it is then backed up with rsync. While it is not safe
generally safe to rely on mtimes and thus nor to backup systems without
--checksum, we should not unnecessarily break things. We came across
this problem because this happens for rpm repos from OpenSUSE that have
extracted files used for installing from a repo, which get broken when
mirroring with rsync, see
https://bugzilla.suse.com/show_bug.cgi?id=1148824 .

This needs openSUSE/obs-build#977 to be
deployed first.

This works together with
rpm-software-management/rpm#2880 . For the full
effect its settings need to be enabled.

First try #58 was
reverted with JanZerebecki@bfa988b .

@JanZerebecki JanZerebecki force-pushed the set-source-date-epoch-try2 branch from be89762 to f4c536b Compare February 6, 2024 11:13
@JanZerebecki
Copy link
Member Author

rpm change was submitted in https://build.opensuse.org/request/show/1143646

@JanZerebecki JanZerebecki marked this pull request as ready for review February 15, 2024 10:52
@JanZerebecki
Copy link
Member Author

I'd appreciate if we could submit this today to ALP (and Factory) before 15:00 UTC to make the submission deadline.

@JanZerebecki
Copy link
Member Author

Arg, one moment, I forgot an edit...

@JanZerebecki JanZerebecki force-pushed the set-source-date-epoch-try2 branch from f4c536b to 843aed9 Compare February 15, 2024 12:17
@JanZerebecki
Copy link
Member Author

Now it is ready to be merged.

replacing rpm its %source_date_epoch_from_changelog . This instead also
increments the date by the build counter obtained from OBS. When
available it uses the mtime that OBS its scmsync obtains from the git
HEAD commit date instead of from the changelog.

It exports the old date in SOURCE_DATE_EPOCH and the new incremented
date in SOURCE_DATE_EPOCH_MTIME. rpm will use
SOURCE_DATE_EPOCH_MTIME for for file mtimes and the old date for
everything else.

All this can be disabled by setting the rpm macro
%disable_obs_set_source_date_epoch to Y .

This uses new variables in /.buildenv that were implemented in
openSUSE/obs-build@5d1da85
.

See
https://reproducible-builds.org/docs/source-date-epoch/#interaction-of-source_date_epoch-with-automatic-rebuilds
for more general discussion.

Increasing the date by the build counter is necessary, so that that even
for rebuilds with unchanged source but updated build depends the date is
always increased, such that no build with a different input/output has
the same date.

Not increasing the date breaks rsync without --checksum or anything else
that relies on modification time stamps of files to detect changes.

When %clamp_mtime_to_source_date_epoch and
%source_date_epoch_from_changelog are enabled and a rebuild is done with
different build dependencies a file may have different content but with
the same mtime. When two releases of this rpm are extracted after each
other at the same location and the extracted files are synced with rsync
without --checksum, then the resulting copy will contain a mix of files
from the old and new build. This can happen on installed systems due to
updates when it is then backed up with rsync. While it is not safe
generally safe to rely on mtimes and thus nor to backup systems without
--checksum, we should not unnecessarily break things. We came across
this problem because this happens for rpm repos from OpenSUSE that have
extracted files used for installing from a repo, which get broken when
mirroring with rsync, see
https://bugzilla.suse.com/show_bug.cgi?id=1148824 .

This needs openSUSE/obs-build#977 to be
deployed first.

This works together with
rpm-software-management/rpm#2880 . For the full
effect its settings need to be enabled.

First try openSUSE#58 was
reverted with bfa988b .
@JanZerebecki JanZerebecki force-pushed the set-source-date-epoch-try2 branch from 843aed9 to d664cd8 Compare February 15, 2024 13:47
@bugfinder bugfinder merged commit a1dbe5c into openSUSE:master Feb 15, 2024
JanZerebecki added a commit to JanZerebecki/pesign-obs-integration that referenced this pull request Feb 16, 2024
otherwise rpm/obs can not extract a source code date to use as
SOURCE_DATE_EPOCH for the build time. Then the build with
pesign-repackage.spec would fail.

See also openSUSE/post-build-checks#62
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants