CI: integration timing results are randomly incomplete

[edsantiago]

Compare two CI logs, one ubuntu-19 and one 20. Search in-page for test path, or reimport, or check dnsname, or many many others (I'm just offering short easy-to-search-for strings). Note that those appear in the bottom timing results section only on one page, not the other, although the tests ran on both.

Something is gobbling up timing results; this seems like a good place to start looking:

podman/test/e2e/common_test.go

Lines 346 to 349 in 9fa09a8

	func processTestResult(f GinkgoTestDescription) {
	tr := testResult{length: f.Duration.Seconds(), name: f.TestText}
	testResults = append(testResults, tr)
	}

(maybe this isn't thread-safe?)

@baude weren't you looking into this some months ago?

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@baude @edsantiago Still an issue?

[edsantiago]

Yes but I seem to be the only one who cares

[vrothberg]

I just hit performance variations in the remote system tests in #8747.

@edsantiago and I looked into a VM together and noticed performance variations on Ubuntu 20.04. #8747 bumps the time out from 60 to 120 seconds.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@edsantiago @vrothberg Since we merged the bump can we close this issue?

[edsantiago]

The issue is not fixed. Example: ubuntu 2010 log for PR 9082. Search in-page for "preserves container". The test is run, but there is no summary line. Compare to ubuntu 2004 log for same PR. Here the test is run and also summarized at bottom.

This is a problem because we still don't understand why ubuntu 2010 runs slower than 2004, and without logged timing results there is no possible way to actually look into the difference.

I really feel like this must be a trivial one-line fix for someone who understands Go arrays and threads.