Skip to content

Commit

Permalink
update mtime to avoid mirror failures
Browse files Browse the repository at this point in the history
Mirrors may use rsync with the skip on same mtime feature, which would
skip files that are different in content but have the same mtime. This
results in an inconsistent mirror.

Avoid this by touching files of extracted content so the mtime is updated.

This is only needed for ouput that is a directory which happens on
REPO_ONLY but not when output is a single file like a disk image or archive.

When the rpm macro %clamp_mtime_to_source_date_epoch is set to Y to
enable reproducible builds, the mtime of files in the rpm will be set to
the date of the last changes entry, but build dependencies that affect
the content may be newer. This is relevant when extracting such an rpm
in REPO_ONLY output for the installer to use. Some mirrors may fail to
sync to the newest content as they skipped them. This would make an
installer using that mirror fail.

Fixes: https://bugzilla.opensuse.org/show_bug.cgi?id=1148824
  • Loading branch information
JanZerebecki committed May 23, 2023
1 parent a15501e commit 19308ef
Show file tree
Hide file tree
Showing 2 changed files with 18 additions and 7 deletions.
8 changes: 4 additions & 4 deletions modules/KIWICollect.pm
Original file line number Diff line number Diff line change
Expand Up @@ -1627,7 +1627,7 @@ sub unpackMetapackages {

$this->logMsg('I', "unpack $packPointer->{localfile} ");
$this->{m_util}->unpac_package(
$packPointer->{localfile}, $tmp
$packPointer->{localfile}, $tmp, $this->{m_proddata}->getVar("REPO_ONLY", 'false') eq "true"
);
# copy content of CD1 ... CD<i> subdirs if exists:
for (1..10) {
Expand Down Expand Up @@ -2179,7 +2179,7 @@ sub collectProducts {
}
$this->logMsg('I',
"Unpacking product release package $i in file $file ".$tmp);
$this->{m_util}->unpac_package($file, $tmp);
$this->{m_util}->unpac_package($file, $tmp, $this->{m_proddata}->getVar("REPO_ONLY", 'false') eq "true");

# get all .prod files
local *D;
Expand Down Expand Up @@ -2372,7 +2372,7 @@ sub unpackModules {
my $pack_file = $this->getBestPackFromRepos($module, $arch)
->{'localfile'};
$this->logMsg('I', "Unpacking $pack_file to $arch_tmp_dir/");
$this->{m_util}->unpac_package($pack_file, "$arch_tmp_dir");
$this->{m_util}->unpac_package($pack_file, "$arch_tmp_dir", $this->{m_proddata}->getVar("REPO_ONLY", 'false') eq "true");
}
}
# copy modules from temp dir to targets
Expand Down Expand Up @@ -2447,7 +2447,7 @@ sub unpackInstSys {
my $pack_file =
$this->getBestPackFromRepos($module, $arch)->{'localfile'};
$this->logMsg('I', "Unpacking $pack_file to $arch_tmp_dir");
$this->{m_util}->unpac_package($pack_file, "$arch_tmp_dir");
$this->{m_util}->unpac_package($pack_file, "$arch_tmp_dir", $this->{m_proddata}->getVar("REPO_ONLY", 'false') eq "true");
}
}
# copy inst_sys_packages from temp dir to targets
Expand Down
17 changes: 14 additions & 3 deletions modules/KIWIUtil.pm
Original file line number Diff line number Diff line change
Expand Up @@ -483,10 +483,12 @@ sub unpac_package {
# $this - class name; always call as member
# $p_uri - uri of the rpm file to unpack
# $dir - target dir for the unpacking, created if necessary
# $touch - do not restore mtime on extracted files, reproducible builds can make changes without newer mtimes, use this when creating directories instead of images like REPO_ONLY to makes sure that rsync skip by time still works
# ---
my $this = shift;
my $p_uri = shift;
my $dir = shift;
my $touch = shift;
my $retval = 0;
if(!($this and $p_uri and $dir)) {
$retval = 1;
Expand All @@ -501,10 +503,14 @@ sub unpac_package {
}
}
if($p_uri =~ m{(.*\.tgz|.*\.tar\.gz|.*\.taz|.*\.tar\.Z)}) {
my $out = qx(cd $dir && tar -zxvfp $p_uri);
my $arg = "";
if($touch) {
$arg = "--touch";
}
my $out = qx(cd $dir && tar -zxvfp $arg $p_uri);
my $status = $?>>8;
if($status != 0) {
my $msg = "[E] command cp $dir && tar xvzfp $p_uri failed!\n";
my $msg = "[E] command cd $dir && tar xvzfp $arg $p_uri failed!\n";
$this->{m_collect}->logMsg("E", $msg);
$this->{m_collect}->logMsg("E", "\t$out\n");
$retval = 5;
Expand All @@ -514,7 +520,12 @@ sub unpac_package {
$this->{m_collect}->logMsg("I", $msg);
}
} elsif($p_uri =~ m{.*\.rpm}i) {
my $out = qx(cd $dir && unrpm -q $p_uri);
my $arg = "";
if($touch) {
$arg = "--no-preserve=time";
}
my $tmpdir = KIWIQX::qxx ("mktemp -qdt kiwiunpac.XXXXXX"); chomp $tmpdir;
my $out = qx(cd $tmpdir && unrpm -q $p_uri && find $tmpdir -mindepth 1 -maxdepth 1 -exec cp -a $arg -t $dir '{}' '+' && rm -rf $tmpdir);
} else {
$this->{m_collect}->logMsg("E", "[E] cannot process file $p_uri\n");
$retval = 4;
Expand Down

0 comments on commit 19308ef

Please sign in to comment.