Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test-pull-summary-sigs.sh fails when run on a tmpfs #2245

Closed
smcv opened this issue Nov 22, 2020 · 1 comment · Fixed by #2246
Closed

test-pull-summary-sigs.sh fails when run on a tmpfs #2245

smcv opened this issue Nov 22, 2020 · 1 comment · Fixed by #2246

Comments

@smcv
Copy link
Contributor

smcv commented Nov 22, 2020

Versions: libostree 2020.8 or git master (commit 024b169 tested); Debian unstable rolling release as of 2020-11-22.

libostree 2020.8 failed to build with test failures on most of Debian's official autobuilders (all except mips*el). I believe this is correlated with the tests being run with /var/tmp on a tmpfs: my understanding is that for architectures where enough RAM or RAM+swap is available, Debian is moving towards doing production builds in a chroot hosted on a tmpfs, in order to make dependency installation and build happen as fast as possible.

I was able to reproduce this by building libostree on a virtual machine with small loopback filesystems mounted on /media/btrfs, /media/ext4 and /media/xfs, and a tmpfs on /media/tmpfs, then running commands like:

make check TESTS=tests/test-pull-summary-sigs.sh TEST_TMPDIR=/media/xfs

The test repeatably passed (5/5 consecutive runs) on btrfs, ext4 and xfs, but repeatably failed (5/5 consecutive runs) on tmpfs.

test-suite.log

@smcv
Copy link
Contributor Author

smcv commented Nov 22, 2020

It appears to be possible to fix this by using touch(1) to ensure that the mtime of the summary and summary.sig changes every time they are overwritten.

smcv added a commit to smcv/ostree that referenced this issue Nov 22, 2020
If this is not done, the test can fail when the temporary directory is
a tmpfs: for example this happens during build-time testing with /var/tmp
on tmpfs or TEST_TMPDIR pointing to a tmpfs, or installed-tests with
gnome-desktop-testing-runner allocating the test directory on a tmpfs.

In particular, many of Debian's official autobuilders now do the entire
build and test procedure in a chroot hosted on a tmpfs, to improve build
performance and prevent fsync overhead.

In this situation, it appears that overwriting summary.sig with a copy
of summary.sig.2 is not sufficient for the web server to tell the
libostree client that it needs to be re-downloaded. I'm not completely
sure why, because tmpfs does appear to have sub-second-resolution
timestamps, but forcing a distinct mtime is certainly enough to
resolve it.

Resolves: ostreedev#2245
Bug-Debian: https://bugs.debian.org/975418
Signed-off-by: Simon McVittie <[email protected]>
raspbian-autopush pushed a commit to raspbian-packages/ostree that referenced this issue Nov 26, 2020
If this is not done, the test can fail when the temporary directory is
a tmpfs: for example this happens during build-time testing with /var/tmp
on tmpfs or TEST_TMPDIR pointing to a tmpfs, or installed-tests with
gnome-desktop-testing-runner allocating the test directory on a tmpfs.

In particular, many of Debian's official autobuilders now do the entire
build and test procedure in a chroot hosted on a tmpfs, to improve build
performance and prevent fsync overhead.

In this situation, it appears that overwriting summary.sig with a copy
of summary.sig.2 is not sufficient for the web server to tell the
libostree client that it needs to be re-downloaded. I'm not completely
sure why, because tmpfs does appear to have sub-second-resolution
timestamps, but forcing a distinct mtime is certainly enough to
resolve it.

Resolves: ostreedev/ostree#2245
Bug-Debian: https://bugs.debian.org/975418
Signed-off-by: Simon McVittie <[email protected]>
Forwarded: ostreedev/ostree#2246

Gbp-Pq: Name test-pull-summary-sigs-Set-timestamps-to-serve-expected-f.patch
raspbian-autopush pushed a commit to raspbian-packages/ostree that referenced this issue Dec 18, 2020
If this is not done, the test can fail when the temporary directory is
a tmpfs: for example this happens during build-time testing with /var/tmp
on tmpfs or TEST_TMPDIR pointing to a tmpfs, or installed-tests with
gnome-desktop-testing-runner allocating the test directory on a tmpfs.

In particular, many of Debian's official autobuilders now do the entire
build and test procedure in a chroot hosted on a tmpfs, to improve build
performance and prevent fsync overhead.

In this situation, it appears that overwriting summary.sig with a copy
of summary.sig.2 is not sufficient for the web server to tell the
libostree client that it needs to be re-downloaded. I'm not completely
sure why, because tmpfs does appear to have sub-second-resolution
timestamps, but forcing a distinct mtime is certainly enough to
resolve it.

Resolves: ostreedev/ostree#2245
Bug-Debian: https://bugs.debian.org/975418
Signed-off-by: Simon McVittie <[email protected]>
Forwarded: ostreedev/ostree#2246

Gbp-Pq: Name test-pull-summary-sigs-Set-timestamps-to-serve-expected-f.patch
dbnicholson pushed a commit to endlessm/ostree that referenced this issue Mar 17, 2021
If this is not done, the test can fail when the temporary directory is
a tmpfs: for example this happens during build-time testing with /var/tmp
on tmpfs or TEST_TMPDIR pointing to a tmpfs, or installed-tests with
gnome-desktop-testing-runner allocating the test directory on a tmpfs.

In particular, many of Debian's official autobuilders now do the entire
build and test procedure in a chroot hosted on a tmpfs, to improve build
performance and prevent fsync overhead.

In this situation, it appears that overwriting summary.sig with a copy
of summary.sig.2 is not sufficient for the web server to tell the
libostree client that it needs to be re-downloaded. I'm not completely
sure why, because tmpfs does appear to have sub-second-resolution
timestamps, but forcing a distinct mtime is certainly enough to
resolve it.

Resolves: ostreedev/ostree#2245
Bug-Debian: https://bugs.debian.org/975418
Signed-off-by: Simon McVittie <[email protected]>
(cherry picked from commit 07c4249)
dbnicholson pushed a commit to endlessm/ostree that referenced this issue Mar 17, 2021
If this is not done, the test can fail when the temporary directory is
a tmpfs: for example this happens during build-time testing with /var/tmp
on tmpfs or TEST_TMPDIR pointing to a tmpfs, or installed-tests with
gnome-desktop-testing-runner allocating the test directory on a tmpfs.

In particular, many of Debian's official autobuilders now do the entire
build and test procedure in a chroot hosted on a tmpfs, to improve build
performance and prevent fsync overhead.

In this situation, it appears that overwriting summary.sig with a copy
of summary.sig.2 is not sufficient for the web server to tell the
libostree client that it needs to be re-downloaded. I'm not completely
sure why, because tmpfs does appear to have sub-second-resolution
timestamps, but forcing a distinct mtime is certainly enough to
resolve it.

Resolves: ostreedev/ostree#2245
Bug-Debian: https://bugs.debian.org/975418
Signed-off-by: Simon McVittie <[email protected]>
(cherry picked from commit 07c4249)
dbnicholson pushed a commit to endlessm/ostree that referenced this issue Mar 19, 2021
If this is not done, the test can fail when the temporary directory is
a tmpfs: for example this happens during build-time testing with /var/tmp
on tmpfs or TEST_TMPDIR pointing to a tmpfs, or installed-tests with
gnome-desktop-testing-runner allocating the test directory on a tmpfs.

In particular, many of Debian's official autobuilders now do the entire
build and test procedure in a chroot hosted on a tmpfs, to improve build
performance and prevent fsync overhead.

In this situation, it appears that overwriting summary.sig with a copy
of summary.sig.2 is not sufficient for the web server to tell the
libostree client that it needs to be re-downloaded. I'm not completely
sure why, because tmpfs does appear to have sub-second-resolution
timestamps, but forcing a distinct mtime is certainly enough to
resolve it.

Resolves: ostreedev/ostree#2245
Bug-Debian: https://bugs.debian.org/975418
Signed-off-by: Simon McVittie <[email protected]>
(cherry picked from commit 07c4249)
raspbian-autopush pushed a commit to raspbian-packages/ostree that referenced this issue Mar 27, 2022
If this is not done, the test can fail when the temporary directory is
a tmpfs: for example this happens during build-time testing with /var/tmp
on tmpfs or TEST_TMPDIR pointing to a tmpfs, or installed-tests with
gnome-desktop-testing-runner allocating the test directory on a tmpfs.

In particular, many of Debian's official autobuilders now do the entire
build and test procedure in a chroot hosted on a tmpfs, to improve build
performance and prevent fsync overhead.

In this situation, it appears that overwriting summary.sig with a copy
of summary.sig.2 is not sufficient for the web server to tell the
libostree client that it needs to be re-downloaded. I'm not completely
sure why, because tmpfs does appear to have sub-second-resolution
timestamps, but forcing a distinct mtime is certainly enough to
resolve it.

Resolves: ostreedev/ostree#2245
Bug-Debian: https://bugs.debian.org/975418
Signed-off-by: Simon McVittie <[email protected]>
Forwarded: ostreedev/ostree#2246

Gbp-Pq: Name test-pull-summary-sigs-Set-timestamps-to-serve-expected-f.patch
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant