From 099bd9eb5bc57ac24ca3a8cd53187cd66a0a6e0f Mon Sep 17 00:00:00 2001 From: Adam Kaplan Date: Fri, 6 Dec 2024 17:38:41 -0500 Subject: [PATCH] OCPBUILD-174: Reresh CONTRIBUTING Guide Clean up the README and contributing guide so it is easier to comprehend and provide fuller context to contributors. The original README was a bare document that was mostly untouched after OpenShift 4.0 was released. Most of the contributors at the time were OpenShift veterans and had deep "head knowledge" of how OpenShift worked; a fully detailed README or contributor guide was not a top priority. This update provides fuller context to contributors, linking out to the openshift-controller-manager repository and Red Hat's official docs for Builds with BuildConfig. The contributor guide is restructured with a more natural developer progression (set up, clone, compile, test, deploy). Deploy instructions are updated to reference cluster-bot for Red Hat employees, and self- managed install modes for the general public. To encourage cluster-bot use, contributors are encouraged to submit pull requests with only unit tests passing; testing code changes end to end is not required prior to PR submission. Signed-off-by: Adam Kaplan --- CONTRIBUTING.md | 203 +++++++++++++++++++++++++++++++----------------- README.md | 33 +++++++- 2 files changed, 160 insertions(+), 76 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index fdeafe7c59..71e6fc1568 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,113 +1,172 @@ -# Contributing to the Openshift Builder +# Contributing Guide -The OpenShift Builder image drives [OpenShift Builds](https://docs.okd.io/latest/dev_guide/builds/index.html), -leveraging [Buildah](https://github.com/containers/buildah) and [Source-To-Image](https://github.com/openshift/source-to-image) -to turn source code into images that can be deployed on Kubernetes. +## Prerequisites -## Getting the code +`openshift/builder` is designed to run exclusively on Linux systems. If you are developing on a +Windows or Mac machine, we recommend setting up a virtual environment that runs the latest Fedora, +CentOS Stream, or Red Hat Enterprise Linux distribution. -To get started, [fork](https://help.github.com/articles/fork-a-repo) the [openshift/builder](https://github.com/openshift/builder) repo. +You will also need to install the following dependencies on your system: -## Developing +- Go SDK - refer to [go.mod](./go.mod) for the minimum required version. +- The following packages: + - `btrfs-progs-devel` + - `device-mapper-devel` + - `gpgme-devel` -### Testing on an OpenShift Cluster +## Compiling the Code -The easiest way to test your changes is to launch an OpenShift 4.x cluster. -First, go to [try.openshift.com](https://try.openshift.com) to obtain a pull secret and download the installer. -Follow the instructions to launch a cluster on AWS. +To get started, [fork](https://help.github.com/articles/fork-a-repo) the +[openshift/builder](https://github.com/openshift/builder) repo, then clone it to a local directory: -If you want the latest `openshift-install` and `oc` clients, go to the [openshift-release](https://openshift-release.svc.ci.openshift.org/) -page, select the channel and build you wish to install, and download the respective `oc` and `openshift-installer` binaries. -There are three types of channels you can obtain the installer from: +```sh +$ mkdir -p ~/go/src/github.com/openshift +$ cd ~/go/src/github.com/openshift +$ git clone https://github.com/openshift/builder.git +$ cd builder +``` -1. `4-stable` - these are stable releases of OpenShift 4, corresponding to GA or beta releases. -2. `4.x.0-nightly` - nightly development releases, with payloads published to quay.io. -3. `4.x.0-ci` - bleeding-edge releases published to the OpenShift CI imagestreams. +Once in the source code directory, run the `build` Makefile target to compile the binary: -**Note**: Installs from the `4.x.0-ci` channel require a pull secret to `registry.ci.openshift.org`, which is only available to Red Hat OpenShift developers. For detailed steps, refer [Obtain Pull Secret for Internal registry](https://github.com/openshift/builder/blob/master/CONTRIBUTING.md#obtaining-pull-secret-for-internal-registry). +```sh +$ make build +``` -After your cluster is installed, you will need to do the following: +If the build succeeds, you are ready to start making code changes! -1. Patch the cluster version so that you can launch your own builder image: +## Submitting a Pull Request -``` -$ oc patch clusterversion/version --patch '{"spec":{"overrides":[{"group":"v1", "kind":"ConfigMap", "namespace":"openshift-controller-manager-operator", "name":"openshift-controller-manager-images", "unmanaged":true}]}}' --type=merge -``` +Before submitting a pull request, ensure all unit tests are passing by running `make test`. You are +encouraged to add your own unit tests alongside code contributions. This repository uses the +standard Go test framework for unit tests. -or, scale down the CVO pod replica: +Once you are satisfied with your code changes (and unit tests are passing), submit a pull request +to the [openshift/builder](https://github.com/openshift/builder) repo. A member of the OpenShift +team will evaluate your change for approval. -``` -$ oc scale --replicas 0 -n openshift-cluster-version deployments/cluster-version-operator -``` +## Testing -2. Make your code changes and build the binary with `make build`. -> While running `make build` if you come across any error something like -` OS_GIT_MAJOR: unbound variable`, make sure that you have pulled all the tags from the [openshift/builder](https://github.com/openshift/builder) repo. - You can use `git fetch upstream --tags` to pull all the tags. +The OpenShift Builder image is intended to run exclusively as a container within an OpenShift +build. The best way to experiment with new code changes is to build the image, push to a container +registry, and then configure OpenShift to deploy your "test" image when running builds. -3. Build the image using the `Dockerfile.dev` file, giving it a unique tag: +Red Hat employees can also use `cluster-bot` (available on the internal Slack as an app) to launch +a cluster that includes code changes submitted in a pull request. Refer to the +[ci-chat-bot](https://github.com/openshift/ci-chat-bot) source for more information. + +### Building the image + +This repository can build two flavors of images: + +1. A "developer" image, which uses publicly available base images. +2. The "product" image, which uses base images only available on Red Hat's CI infrastructure. + +The two images are intended to be similar, but may have some skews due to differences in how +OpenShift's build system assembles the entire platform (aka "the payload"). + +#### Developer Image + +Run the `build-devel-image` Makefile target, giving it an appropriate host, repository, and image tag: ``` $ make build-devel-image IMAGE=/ TAG= ``` -or if you are using `buildah`: +#### Product Image -``` -$ buildah bud -t /: -f Dockerfile.dev . -``` +This procedure will only succeed if you are a Red Hat employee and have been granted permission to +pull images from the OpenShift CI system. Refer to the +[Openshift CI docs](https://docs.ci.openshift.org/docs/how-tos/use-registries-in-build-farm/#how-do-i-gain-access-to-qci) +for instructions on how to request access to the CI container registries. -4. Push the image to a registry accessible from the cluster (e.g. your repository on quay.io). -5. Patch the ConfigMap in the override above to instruct the cluster to use your builder image: +1. First, log into the [api.ci](https://console-openshift-console.apps.ci.l2s4.p1.openshiftapps.com/) + cluster using your Red Hat SSO credentials. +2. Use the `Copy Login Command` to get command line access to the cluster. Note that you must have + the `oc` command line installed on your system: -``` -$ oc patch configmap openshift-controller-manager-images -n openshift-controller-manager-operator --patch '{"data":{"builderImage":"/:"}}' --type=merge -``` + ```sh + $ oc login --token= --server=: + ``` -6. Watch the openshift controller manager pods rollout (this can take a few minutes): +3. Once logged in to the cluster successfully, use `oc` to authenticate with the cluster's + container registry (`registry.ci.openshift.org`): -``` -$ oc get ds controller-manager -n openshift-controller-manager -w -``` + ```sh + $ oc registry login + ``` -7. Trigger an OpenShift build via `oc start-build`. You can use one of the templates suggested in `oc new-app` to populate your project with a build. +4. Finally, build the container image using the `build-image` Makefile target: -8. To set your cluster back to its original state, either remove the `overrides` section added in step 1, or, scale up the CVO pods back to previous count: + ```sh + $ make build IMAGE=/ TAG= + ``` -``` -$ oc scale --replicas 1 -n openshift-cluster-version deployments/cluster-version-operator +### Testing on an OpenShift Cluster -``` +#### Cluster-bot -### Obtaining Pull Secret for Internal registry +For Red Hat employees, the easiest way to test code changes is to submit a pull request (add "WIP" +to the title if your code is not ready to merge), then create a cluster with `cluster-bot` +referencing your PR number. Refer to the +[cluster-bot FAQ](https://github.com/openshift/ci-chat-bot/blob/main/docs/FAQ.md#frequently-asked-questions) +for an example of how to do this. -1. Connect to the VPN, and log in using Internal SSO to the [OpenShift Cluster](https://console-openshift-console.apps.ci.l2s4.p1.openshiftapps.com/). +#### Self-Managed OpenShift -2. Use `Copy Login Command` to login to the cluster using the `token`: +For the general public, you must first gain access to a _self-managed_ OpenShift cluster. The +builder image cannot be tested on managed clusters, such as Red Hat OpenShift Service on AWS +(ROSA), because the procedure for testing brings your cluster into an unsupported state. Options +include self-managed installation paths on [console.redhat.com](https://console.redhat.com) or +running [OpenShift Local](https://developers.redhat.com/products/openshift-local/overview) on your +machine. -``` -$ oc login --token= --server=: -``` +You may also deploy OpenShift from a CI payload - refer to the appropriate +[release status page](https://docs.ci.openshift.org/docs/getting-started/useful-links/#services) +for instructions on how to deploy a cluster with a specific channel and payload. -3. Once logged in to the cluster successfully, login to the [registry](registry.ci.openshift.org): +After your cluster is installed, you will need to do the following: -``` -$ oc registry login -``` +1. Patch the cluster version so that you can launch your own builder image: -4. Now, you should be able to pull the images from `registry.ci.openshift.org`. -5. You can also build the image now using `Dockerfile.rhel8` in `step 3` of [Testing on an OpenShift Cluster](https://github.com/openshift/builder/blob/master/CONTRIBUTING.md#testing-on-an-openshift-cluster). + ```sh + $ oc patch clusterversion/version --patch '{"spec":{"overrides":[{"group":"v1", "kind":"ConfigMap", "namespace":"openshift-controller-manager-operator", "name":"openshift-controller-manager-images", "unmanaged":true}]}}' --type=merge + ``` -## Submitting a Pull Request +2. Alternatively, scale the Cluster Version Operator (CVO) deployment down to zero: + + ```sh + $ oc scale --replicas 0 -n openshift-cluster-version deployments/cluster-version-operator + ``` + +3. Patch the ConfigMap in the override above to instruct the cluster to use your builder image: + + ```sh + $ oc patch configmap openshift-controller-manager-images -n openshift-controller-manager-operator --patch '{"data":{"builderImage":"/:"}}' --type=merge + ``` + +4. Wait for the openshift controller manager deployment to roll out the update (this can take a few + minutes): + + ```sh + $ oc get deployment controller-manager -n openshift-controller-manager -w + ``` + +5. To test your changes end to end, trigger an OpenShift build via `oc start-build`. You can use + one of the templates suggested in `oc new-app` to populate your project with a build. + +6. To set your cluster back to its original state, either remove the `overrides` section added in + step 1, or scale the CVO deployment back to previous count: -Once you are satisfied with your code changes, you may submit a pull request to the [openshift/builder](https://github.com/openshift/builder) repo. -A member of the OpenShift Developer Experience team will evaluate your change for approval. + ```sh + $ oc scale --replicas 1 -n openshift-cluster-version deployments/cluster-version-operator + ``` -## End-to-End Testing +### E2E Test Suite -The OpenShift builder is considered a core component of the OpenShift Container Platform. -The e2e test suite run against each pull request is located in the [openshift/origin](https://github.com/openshift/origin) repo. -Tests whose description contains `[Feature:Builds]` are run as part of the e2e suite, and are generally placed in the -`test/extended/builds` directory. -Refer to the extended tests [README](https://github.com/openshift/origin/blob/master/test/extended/README.md) -for instructions on how to develop and run these tests against your own cluster. +The OpenShift builder is considered a core component of the OpenShift Container Platform. The e2e +test suite run against each pull request is located in the +[openshift/origin](https://github.com/openshift/origin) repo. Tests whose description contains +`[Feature:Builds]` are run as part of the e2e suite, and are generally placed in the +`test/extended/builds` directory. Refer to the extended tests +[README](https://github.com/openshift/origin/blob/master/test/extended/README.md) for instructions +on how to develop and run these tests against your own cluster. diff --git a/README.md b/README.md index d650e8e82d..28e517e273 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,31 @@ -# builder -The image run by build pods to execute image building+pushing +# OpenShift Builder + +The OpenShift Builder container image drives OpenShift's +[Builds using BuildConfig](https://docs.redhat.com/en/documentation/openshift_container_platform/4.17/html/builds_using_buildconfig/index), +leveraging [Buildah](https://github.com/containers/buildah) and +[Source-To-Image](https://github.com/openshift/source-to-image) to turn source code into images +that can be deployed on Kubernetes. The container image also handles the lifecycle of the build, +from obtaining source code to pushing the output image to a container registry. + +Much of the work here and in the build controllers for +[openshift-controller-manager](https://github.com/openshift/openshift-controller-manager) inspired +the CNCF [Shipwright](https://shipwright.io) project. + +## Getting Started + +The OpenShift Builder container image is intended to run exclusively as a container within an +OpenShift build pod. The best way to run the container image is to follow the official documentation +on how to run a build with +[BuildConfigs](https://docs.redhat.com/en/documentation/openshift_container_platform/4.17/html/builds_using_buildconfig/understanding-image-builds). + +## Contributing -### Looking for contribution guidelines? Glad to have you here! -Please check [CONTRIBUTING.md](https://github.com/openshift/builder/blob/master/CONTRIBUTING.md) for detailed steps to develop & test the changes. + +Please check the [CONTRIBUTING guide](./CONTRIBUTING.md) for detailed steps on how to develop and +test your code. + +## License + +Licensed under Apache License, Version 2.0. Refer to [LICENSE](./LICENSE) for full terms and +conditions.