Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: Add PR number to build info and share test caches #219

Merged
merged 6 commits into from
Oct 24, 2023

Conversation

achilleas-k
Copy link
Member

This PR makes two small changes to the test pipelines:

  1. Adds the PR number to the build info when a new image is built. This number is printed when filtering builds to make it easier to determine why image builds are skipped and it's also used to list images build in all pipelines of a given PR in the check-build-coverage jobs. Eventually, we can use this information to add messages to the PR so it's easier to track what is being built by each pipeline run.
  2. Puts the rpmmd and build info cache in the working directory and configures build pipeline generators to share that cache. This should speed up test generation. It'll be especially good for speeding up depsolving for manifest generation.

When filtering builds for a CI run and the commit ID is printed for an
image that was already built, also print the PR number (and URL) if it's
set.
The same way we print a list of image configurations built in a specific
CI run, let's also print a list of configurations built in the same PR
(identified by the CI_COMMIT_BRANCH).

Again, this makes it a bit more convenient to check which images were
rebuilt in a PR, but we should eventually use this information to inform
PR authors and reviewers of changes in image definitions.
These have already been set in the repository config.
Set the cache root to be under the working directory.  This is used for
rpmmd and the build info downloaded from S3.  By putting it in the
working directory, we can share the cache with subsequent pipelines to
speed up subsequent runs.
The test cache holds the RPM metadata for depsolving (rpmmd) and a local
copy of the build info from s3.  Caching it and sharing it between jobs
should speed up config generation.
@achilleas-k achilleas-k force-pushed the test/build-info/pr-number branch from 2728ea3 to 048ecfb Compare October 20, 2023 21:07
Copy link
Contributor

@ochosi ochosi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't judge how the cache bits work, but the PR info looks good! Thank you so much for this - CI speedups ftw ❤️

@achilleas-k
Copy link
Member Author

Looking at the numbers of the jobs, it doesn't seem to have had much of an effect. Maybe there's a smarter way to do it, but I think it doesn't hurt to keep it.

@achilleas-k
Copy link
Member Author

I want to add a starter job that pulls in all the info.json files from the build cache and then shares it with all the following pipelines through the cache. That should make things a bit faster. I'll do that in a followup because I'd like to get the PR number part in for jobs to start adding it.

@achilleas-k achilleas-k added this pull request to the merge queue Oct 24, 2023
Merged via the queue into osbuild:main with commit e6526c2 Oct 24, 2023
7 checks passed
@achilleas-k achilleas-k deleted the test/build-info/pr-number branch October 24, 2023 15:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants