Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get release and image info from a single-source-of-truth #178

Merged
merged 1 commit into from
Aug 30, 2022

Conversation

cevich
Copy link
Member

@cevich cevich commented Aug 23, 2022

Fixes: #135

Previously, a maintainer integrating a new OS release was required to
update multiple files. Since all images are produced from make targets,
simple variables can serve as a single-source-of-truth. Update the
Makefile and image build configurations to accept these values at
build time.

Also do some minor re-arranging, code cleanup, and comment updates.

@cevich cevich requested a review from lsm5 August 23, 2022 19:19
@cevich cevich force-pushed the release_ver_from_yaml branch from b020e46 to 0d5c6de Compare August 23, 2022 19:24
Copy link
Member

@lsm5 lsm5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM after happy tests. This is really convenient!!

@cevich
Copy link
Member Author

cevich commented Aug 23, 2022

after happy tests

Right...that's almost always the actual hard part 🤣

@cevich cevich force-pushed the release_ver_from_yaml branch from 0d5c6de to 289cc6a Compare August 23, 2022 19:48
@cevich
Copy link
Member Author

cevich commented Aug 23, 2022

Force-pushed: More Makefile updates - I missed a few more base_images/cloud.yml dependencies.

@cevich cevich force-pushed the release_ver_from_yaml branch 5 times, most recently from 3960e39 to b5b5395 Compare August 24, 2022 21:25
cache_images/cloud.yml Outdated Show resolved Hide resolved
@cevich cevich force-pushed the release_ver_from_yaml branch from b5b5395 to 9726130 Compare August 24, 2022 21:58
@github-actions
Copy link

[Cirrus CI build](https://cirrus-ci.com/build/${{ steps.retro.outputs.bid }}) successful. Found built image names and IDs:

Stage Image Name IMAGE_SUFFIX
base fedora b4746025087795200
base fedora-aws b4746025087795200
base fedora-aws-arm64 b4746025087795200
base image-builder b4746025087795200
base prior-fedora b4746025087795200
base ubuntu b4746025087795200
cache build-push c4746025087795200
cache fedora c4746025087795200
cache fedora-aws c4746025087795200
cache fedora-netavark c4746025087795200
cache fedora-netavark-aws-arm64 c4746025087795200
cache fedora-podman-aws-arm64 c4746025087795200
cache fedora-podman-py c4746025087795200
cache prior-fedora c4746025087795200
cache ubuntu c4746025087795200

@cevich cevich force-pushed the release_ver_from_yaml branch from 9726130 to a6e1030 Compare August 25, 2022 15:09
cevich added a commit to cevich/automation_images that referenced this pull request Aug 25, 2022
***DEPENDS ON:***
containers#178

Downstream CI needs dictate early testing on new Fedora releases at the
Beta stage or earlier.  Unfortunately at the time of this commit, the
Fedora cloud-sig does not provide ready-made beta images in AWS EC2.

Add a new image-build stage called `import_images`, that takes care of
bringing the generic images into AWS.  Update the `base_images` stage
such that it will locate and use the imported images.  Unify all the
build-stage wrapper scripts into a single/common `ci/make.sh` wrapper.

Finally, make a few semi-related comment/readability improvements in
various places.

Signed-off-by: Chris Evich <[email protected]>
@cevich cevich changed the title Use base_image/cloud.yml as release version SoT Get release and image info from a single-source-of-truth Aug 25, 2022
@cevich cevich force-pushed the release_ver_from_yaml branch 5 times, most recently from eba99e1 to b66df1a Compare August 25, 2022 20:10
@cevich cevich marked this pull request as draft August 25, 2022 20:13
@cevich cevich force-pushed the release_ver_from_yaml branch from b66df1a to e4f8b9d Compare August 25, 2022 20:42
cevich added a commit to cevich/automation_images that referenced this pull request Aug 25, 2022
***DEPENDS ON:***
containers#178

Downstream CI needs dictate early testing on new Fedora releases at the
Beta stage or earlier.  Unfortunately at the time of this commit, the
Fedora cloud-sig does not provide ready-made beta images in AWS EC2.

Add a new image-build stage called `import_images`, that takes care of
bringing the generic images into AWS.  Update the `base_images` stage
such that it will locate and use the imported images.  Unify all the
build-stage wrapper scripts into a single/common `ci/make.sh` wrapper.

Finally, make a few semi-related comment/readability improvements in
various places.

Signed-off-by: Chris Evich <[email protected]>
@cevich cevich force-pushed the release_ver_from_yaml branch from e4f8b9d to c87aa18 Compare August 25, 2022 21:20
cevich added a commit to cevich/automation_images that referenced this pull request Aug 25, 2022
***DEPENDS ON:***
containers#178

Downstream CI needs dictate early testing on new Fedora releases at the
Beta stage or earlier.  Unfortunately at the time of this commit, the
Fedora cloud-sig does not provide ready-made beta images in AWS EC2.

Add a new image-build stage called `import_images`, that takes care of
bringing the generic images into AWS.  Update the `base_images` stage
such that it will locate and use the imported images.  Unify all the
build-stage wrapper scripts into a single/common `ci/make.sh` wrapper.

Finally, make a few semi-related comment/readability improvements in
various places.

Signed-off-by: Chris Evich <[email protected]>
cevich added a commit to cevich/automation_images that referenced this pull request Aug 25, 2022
***DEPENDS ON:***
containers#178

Downstream CI needs dictate early testing on new Fedora releases at the
Beta stage or earlier.  Unfortunately at the time of this commit, the
Fedora cloud-sig does not provide ready-made beta images in AWS EC2.

Add a new image-build stage called `import_images`, that takes care of
bringing the generic images into AWS.  Update the `base_images` stage
such that it will locate and use the imported images.  Unify all the
build-stage wrapper scripts into a single/common `ci/make.sh` wrapper.

Finally, make a few semi-related comment/readability improvements in
various places.

Signed-off-by: Chris Evich <[email protected]>
@github-actions
Copy link

Cirrus CI build successful. Found built image names and IDs:

Stage Image Name IMAGE_SUFFIX
base fedora b5823947156488192
base fedora-aws b5823947156488192
base fedora-aws-arm64 b5823947156488192
base image-builder b5823947156488192
base prior-fedora b5823947156488192
base ubuntu b5823947156488192
cache build-push c5823947156488192
cache fedora c5823947156488192
cache fedora-aws c5823947156488192
cache fedora-netavark c5823947156488192
cache fedora-netavark-aws-arm64 c5823947156488192
cache fedora-podman-aws-arm64 c5823947156488192
cache fedora-podman-py c5823947156488192
cache prior-fedora c5823947156488192
cache ubuntu c5823947156488192

cevich added a commit to cevich/automation_images that referenced this pull request Aug 26, 2022
***DEPENDS ON:***
containers#178

Downstream CI needs dictate early testing on new Fedora releases at the
Beta stage or earlier.  Unfortunately at the time of this commit, the
Fedora cloud-sig does not provide ready-made beta images in AWS EC2.

Add a new image-build stage called `import_images`, that takes care of
bringing the generic images into AWS.  Update the `base_images` stage
such that it will locate and use the imported images.  Unify all the
build-stage wrapper scripts into a single/common `ci/make.sh` wrapper.

Finally, make a few semi-related comment/readability improvements in
various places.

Signed-off-by: Chris Evich <[email protected]>
Makefile Outdated Show resolved Hide resolved
cevich added a commit to cevich/automation_images that referenced this pull request Aug 29, 2022
***DEPENDS ON:***
containers#178

Downstream CI needs dictate early testing on new Fedora releases at the
Beta stage or earlier.  Unfortunately at the time of this commit, the
Fedora cloud-sig does not provide ready-made beta images in AWS EC2.

Add a new image-build stage called `import_images`, that takes care of
bringing the generic images into AWS.  Update the `base_images` stage
such that it will locate and use the imported images.  Unify all the
build-stage wrapper scripts into a single/common `ci/make.sh` wrapper.

Finally, make a few semi-related comment/readability improvements in
various places.

Signed-off-by: Chris Evich <[email protected]>
cevich added a commit to cevich/automation_images that referenced this pull request Aug 29, 2022
***DEPENDS ON:***
containers#178

Downstream CI needs dictate early testing on new Fedora releases at the
Beta stage or earlier.  Unfortunately at the time of this commit, the
Fedora cloud-sig does not provide ready-made beta images in AWS EC2.

Add a new image-build stage called `import_images`, that takes care of
bringing the generic images into AWS.  Update the `base_images` stage
such that it will locate and use the imported images.  Unify all the
build-stage wrapper scripts into a single/common `ci/make.sh` wrapper.

Finally, make a few semi-related comment/readability improvements in
various places.

Signed-off-by: Chris Evich <[email protected]>
@cevich cevich force-pushed the release_ver_from_yaml branch from c87aa18 to 3bdde2d Compare August 29, 2022 19:26
@cevich
Copy link
Member Author

cevich commented Aug 29, 2022

Force-push: Simplify, define build variables in the Makefile. Also, added a definition for the CentOS-stream release (used in a few containers and the image_builder image).

cevich added a commit to cevich/automation_images that referenced this pull request Aug 29, 2022
***DEPENDS ON:***
containers#178

Downstream CI needs dictate early testing on new Fedora releases at the
Beta stage or earlier.  Unfortunately at the time of this commit, the
Fedora cloud-sig does not provide ready-made beta images in AWS EC2.

Add a new image-build stage called `import_images`, that takes care of
bringing the generic F37 cloud images into AWS.  Update the
`base_images` stage such that it will locate and use the imported
images.  Unify all the build-stage wrapper scripts into a
single/common `ci/make.sh` wrapper.

Finally, make a few semi-related comment/readability improvements in
various places.

Signed-off-by: Chris Evich <[email protected]>
cevich added a commit to cevich/automation_images that referenced this pull request Aug 29, 2022
***DEPENDS ON:***
containers#178

Downstream CI needs dictate early testing on new Fedora releases at the
Beta stage or earlier.  Unfortunately at the time of this commit, the
Fedora cloud-sig does not provide ready-made beta images in AWS EC2.

Add a new image-build stage called `import_images`, that takes care of
bringing the generic F37 cloud images into AWS.  Update the
`base_images` stage such that it will locate and use the imported
images.  Unify all the build-stage wrapper scripts into a
single/common `ci/make.sh` wrapper.

Finally, make a few semi-related comment/readability improvements in
various places.

Signed-off-by: Chris Evich <[email protected]>
@github-actions
Copy link

Cirrus CI build successful. Found built image names and IDs:

Stage Image Name IMAGE_SUFFIX
base fedora b4713699385540608
base fedora-aws b4713699385540608
base fedora-aws-arm64 b4713699385540608
base image-builder b4713699385540608
base prior-fedora b4713699385540608
base ubuntu b4713699385540608
cache build-push c4713699385540608
cache fedora c4713699385540608
cache fedora-aws c4713699385540608
cache fedora-netavark c4713699385540608
cache fedora-netavark-aws-arm64 c4713699385540608
cache fedora-podman-aws-arm64 c4713699385540608
cache fedora-podman-py c4713699385540608
cache prior-fedora c4713699385540608
cache ubuntu c4713699385540608

Copy link
Member

@edsantiago edsantiago left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Way over my head. I don't understand the AWS_SHARED_CREDENTIALS_FILE stuff, nor why URLs changed from /build to /task, ... but from what little I can understand, I have a few questions

.gitignore Outdated Show resolved Hide resolved
base_images/cloud.yml Outdated Show resolved Hide resolved
cache_images/cloud.yml Outdated Show resolved Hide resolved
Fixes: containers#135

Previously, a maintainer integrating a new OS release was required to
update multiple files.  Since all images are produced from make targets,
simple variables can serve as a single-source-of-truth.  Update the
`Makefile` and image build configurations to accept these values at
build time.

Also do some minor re-arranging, code cleanup, and comment updates.

Signed-off-by: Chris Evich <[email protected]>
@cevich
Copy link
Member Author

cevich commented Aug 30, 2022

why URLs changed from /build to /task

Oh, this is to assist maintainers when viewing images on AWS or GCE, they have an immediate link right to the build-logs (task) for the image, rather than the (less immediately useful) build (collection of tasks). Also note, it's similar to the recently updated table of image names and IDs (comment-bot) - it now contains links to the cirrus-task logs for each image in addition to the (less useful) build.

I don't understand the AWS_SHARED_CREDENTIALS_FILE stuff

The AWS_SHARED_CREDENTIALS_FILE: notused addition is needed because the packer_build function checks the value is non-empty - which is useful for every image build except the image_builder target (and image).

@cevich cevich force-pushed the release_ver_from_yaml branch from 3bdde2d to 7e3a0c2 Compare August 30, 2022 17:16
@github-actions
Copy link

Cirrus CI build successful. Found built image names and IDs:

Stage Image Name IMAGE_SUFFIX
base fedora b5980126226677760
base fedora-aws b5980126226677760
base fedora-aws-arm64 b5980126226677760
base image-builder b5980126226677760
base prior-fedora b5980126226677760
base ubuntu b5980126226677760
cache build-push c5980126226677760
cache fedora c5980126226677760
cache fedora-aws c5980126226677760
cache fedora-netavark c5980126226677760
cache fedora-netavark-aws-arm64 c5980126226677760
cache fedora-podman-aws-arm64 c5980126226677760
cache fedora-podman-py c5980126226677760
cache prior-fedora c5980126226677760
cache ubuntu c5980126226677760

cevich added a commit to cevich/automation_images that referenced this pull request Aug 30, 2022
***DEPENDS ON:***
containers#178

Downstream CI needs dictate early testing on new Fedora releases at the
Beta stage or earlier.  Unfortunately at the time of this commit, the
Fedora cloud-sig does not provide ready-made beta images in AWS EC2.

Add a new image-build stage called `import_images`, that takes care of
bringing the generic F37 cloud images into AWS.  Update the
`base_images` stage such that it will locate and use the imported
images.  Unify all the build-stage wrapper scripts into a
single/common `ci/make.sh` wrapper.

Finally, make a few semi-related comment/readability improvements in
various places.

Signed-off-by: Chris Evich <[email protected]>
cevich added a commit to cevich/automation_images that referenced this pull request Aug 30, 2022
***DEPENDS ON:***
containers#178

Downstream CI needs dictate early testing on new Fedora releases at the
Beta stage or earlier.  Unfortunately at the time of this commit, the
Fedora cloud-sig does not provide ready-made beta images in AWS EC2.

Add a new image-build stage called `import_images`, that takes care of
bringing the generic F37 cloud images into AWS.  Update the
`base_images` stage such that it will locate and use the imported
images.  Unify all the build-stage wrapper scripts into a
single/common `ci/make.sh` wrapper.

Finally, make a few semi-related comment/readability improvements in
various places.

Signed-off-by: Chris Evich <[email protected]>
@cevich
Copy link
Member Author

cevich commented Aug 30, 2022

This seems to be working well, merging.

@cevich cevich merged commit 74f45b9 into containers:main Aug 30, 2022
cevich added a commit to cevich/automation_images that referenced this pull request Aug 31, 2022
***DEPENDS ON:***
containers#178

Downstream CI needs dictate early testing on new Fedora releases at the
Beta stage or earlier.  Unfortunately at the time of this commit, the
Fedora cloud-sig does not provide ready-made beta images in AWS EC2.

Add a new image-build stage called `import_images`, that takes care of
bringing the generic F37 cloud images into AWS.  Update the
`base_images` stage such that it will locate and use the imported
images.  Unify all the build-stage wrapper scripts into a
single/common `ci/make.sh` wrapper.

Finally, make a few semi-related comment/readability improvements in
various places.

Signed-off-by: Chris Evich <[email protected]>
cevich added a commit to cevich/automation_images that referenced this pull request Aug 31, 2022
***DEPENDS ON:***
containers#178

Downstream CI needs dictate early testing on new Fedora releases at the
Beta stage or earlier.  Unfortunately at the time of this commit, the
Fedora cloud-sig does not provide ready-made beta images in AWS EC2.

Add a new image-build stage called `import_images`, that takes care of
bringing the generic F37 cloud images into AWS.  Update the
`base_images` stage such that it will locate and use the imported
images.  Unify all the build-stage wrapper scripts into a
single/common `ci/make.sh` wrapper.

Finally, make a few semi-related comment/readability improvements in
various places.

Signed-off-by: Chris Evich <[email protected]>
cevich added a commit to cevich/automation_images that referenced this pull request Aug 31, 2022
***DEPENDS ON:***
containers#178

Downstream CI needs dictate early testing on new Fedora releases at the
Beta stage or earlier.  Unfortunately at the time of this commit, the
Fedora cloud-sig does not provide ready-made beta images in AWS EC2.

Add a new image-build stage called `import_images`, that takes care of
bringing the generic F37 cloud images into AWS.  Update the
`base_images` stage such that it will locate and use the imported
images.  Unify all the build-stage wrapper scripts into a
single/common `ci/make.sh` wrapper.

Finally, make a few semi-related comment/readability improvements in
various places.

Signed-off-by: Chris Evich <[email protected]>
cevich added a commit to cevich/automation_images that referenced this pull request Aug 31, 2022
***DEPENDS ON:***
containers#178

Downstream CI needs dictate early testing on new Fedora releases at the
Beta stage or earlier.  Unfortunately at the time of this commit, the
Fedora cloud-sig does not provide ready-made beta images in AWS EC2.

Add a new image-build stage called `import_images`, that takes care of
bringing the generic F37 cloud images into AWS.  Update the
`base_images` stage such that it will locate and use the imported
images.  Unify all the build-stage wrapper scripts into a
single/common `ci/make.sh` wrapper.

Finally, make a few semi-related comment/readability improvements in
various places.

Signed-off-by: Chris Evich <[email protected]>
cevich added a commit to cevich/automation_images that referenced this pull request Sep 6, 2022
***DEPENDS ON:***
containers#178

Downstream CI needs dictate early testing on new Fedora releases at the
Beta stage or earlier.  Unfortunately at the time of this commit, the
Fedora cloud-sig does not provide ready-made beta images in AWS EC2.

Add a new image-build stage called `import_images`, that takes care of
bringing the generic F37 cloud images into AWS.  Update the
`base_images` stage such that it will locate and use the imported
images.  Unify all the build-stage wrapper scripts into a
single/common `ci/make.sh` wrapper.

Finally, make a few semi-related comment/readability improvements in
various places.

Signed-off-by: Chris Evich <[email protected]>
cevich added a commit to cevich/automation_images that referenced this pull request Sep 8, 2022
***DEPENDS ON:***
containers#178

Downstream CI needs dictate early testing on new Fedora releases at the
Beta stage or earlier.  Unfortunately at the time of this commit, the
Fedora cloud-sig does not provide ready-made beta images in AWS EC2.

Add a new image-build stage called `import_images`, that takes care of
bringing the generic F37 cloud images into AWS.  Update the
`base_images` stage such that it will locate and use the imported
images.  Unify all the build-stage wrapper scripts into a
single/common `ci/make.sh` wrapper.

Finally, make a few semi-related comment/readability improvements in
various places.

Signed-off-by: Chris Evich <[email protected]>
cevich added a commit to cevich/automation_images that referenced this pull request Sep 8, 2022
***DEPENDS ON:***
containers#178

Downstream CI needs dictate early testing on new Fedora releases at the
Beta stage or earlier.  Unfortunately at the time of this commit, the
Fedora cloud-sig does not provide ready-made beta images in AWS EC2.

Add a new image-build stage called `import_images`, that takes care of
bringing the generic F37 cloud images into AWS.  Update the
`base_images` stage such that it will locate and use the imported
images.  Unify all the build-stage wrapper scripts into a
single/common `ci/make.sh` wrapper.

Finally, make a few semi-related comment/readability improvements in
various places.

Signed-off-by: Chris Evich <[email protected]>
cevich added a commit to cevich/automation_images that referenced this pull request Sep 8, 2022
***DEPENDS ON:***
containers#178

Downstream CI needs dictate early testing on new Fedora releases at the
Beta stage or earlier.  Unfortunately at the time of this commit, the
Fedora cloud-sig does not provide ready-made beta images in AWS EC2.

Add a new image-build stage called `import_images`, that takes care of
bringing the generic F37 cloud images into AWS.  Update the
`base_images` stage such that it will locate and use the imported
images.  Unify all the build-stage wrapper scripts into a
single/common `ci/make.sh` wrapper.

Finally, make a few semi-related comment/readability improvements in
various places.

Signed-off-by: Chris Evich <[email protected]>
cevich added a commit to cevich/automation_images that referenced this pull request Sep 9, 2022
***DEPENDS ON:***
containers#178

Downstream CI needs dictate early testing on new Fedora releases at the
Beta stage or earlier.  Unfortunately at the time of this commit, the
Fedora cloud-sig does not provide ready-made beta images in AWS EC2.

Add a new image-build stage called `import_images`, that takes care of
bringing the generic F37 cloud images into AWS.  Update the
`base_images` stage such that it will locate and use the imported
images.  Unify all the build-stage wrapper scripts into a
single/common `ci/make.sh` wrapper.

Finally, make a few semi-related comment/readability improvements in
various places.

Signed-off-by: Chris Evich <[email protected]>
cevich added a commit to cevich/automation_images that referenced this pull request Sep 12, 2022
***DEPENDS ON:***
containers#178

Downstream CI needs dictate early testing on new Fedora releases at the
Beta stage or earlier.  Unfortunately at the time of this commit, the
Fedora cloud-sig does not provide ready-made beta images in AWS EC2.

Add a new image-build stage called `import_images`, that takes care of
bringing the generic F37 cloud images into AWS.  Update the
`base_images` stage such that it will locate and use the imported
images.  Unify all the build-stage wrapper scripts into a
single/common `ci/make.sh` wrapper.

Finally, make a few semi-related comment/readability improvements in
various places.

Signed-off-by: Chris Evich <[email protected]>
cevich added a commit to cevich/automation_images that referenced this pull request Sep 12, 2022
***DEPENDS ON:***
containers#178

Downstream CI needs dictate early testing on new Fedora releases at the
Beta stage or earlier.  Unfortunately at the time of this commit, the
Fedora cloud-sig does not provide ready-made beta images in AWS EC2.

Add a new image-build stage called `import_images`, that takes care of
bringing the generic F37 cloud images into AWS.  Update the
`base_images` stage such that it will locate and use the imported
images.  Unify all the build-stage wrapper scripts into a
single/common `ci/make.sh` wrapper.

Finally, make a few semi-related comment/readability improvements in
various places.

Signed-off-by: Chris Evich <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Re-use podman/*_release contents in packer
3 participants