kubernetes-sigs · k8s-ci-robot · Jul 8, 2020 · Jun 2, 2020
diff --git a/README.md b/README.md
@@ -22,6 +22,10 @@ Not everything will need a cluster addon. Not everyone will want to use an opera
 
 The lifecycle of a cluster addon is managed alongside the lifecycle of the cluster. Typically it has to be upgraded/downgraded when you move to a newer Kubernetes version. We want to use operators for this: a CRD describes the addon, and then the code which installs whatever the addon does, controlled by the CRD.
 
+> How do I build my own cluster addon operator?
+
+We have created a tutorial on how to create your own addon operator [here](https://github.com/kubernetes-sigs/cluster-addons/tree/master/walkthrough.md)
+
 > What's your current agenda and timeline?
 
 We

diff --git a/coredns/README.md b/coredns/README.md
@@ -1,77 +1,133 @@
-# Example operator for CoreDNS
+> Note: The CoreDNS operator is currently considered as `alpha`, and it is NOT recommended to be used in production.
+
+# CoreDNS Operator
+
+The CoreDNS Operator has been built to enable users to install the [CoreDNS](https://coredns.io) addon 
+on their Kubernetes clusters
+
+## Usage
+
+The CoreDNS operator installs CoreDNS on the cluster helps to manage its resources
+All the resources are installed via the use of `Kustomize`
+This allows us to install the CoreDNS ConfigMap using the `configMapGenerator`, hashing the ConfigMap, 
+which allows the CoreDNS deployment to undergo a proper and safe RollingUpdate
+
+
+One of the main functionality of the operator is to be constantly watching the CoreDNS resources (deployment, ConfigMap, service etc.) and ensuring that it is in a functioning state. 
+Any modification to the CoreDNS resources will result in the operator to reconcile and revert the changes
+
+If there are any changes that is desired in CoreDNS, it can be done via the CoreDNS Custom Resource(CR)
+The CR defines all the necessary specifications required by CoreDNS (example: CoreDNS Version, DNS Domain, Cluster IP and Corefile)
+
+An example CR is as follows:
+
+```yaml
+apiVersion: addons.x-k8s.io/v1alpha1
+kind: CoreDNS
+metadata:
+  name: coredns-operator
+  namespace: kube-system
+spec:
+  version: 1.3.1
+  dnsDomain: cluster.local
+  dnsIP: 10.96.0.10
+  corefile: |
+    .:53 {
+        errors
+        health
+        kubernetes cluster.local in-addr.arpa ip6.arpa {
+           pods insecure
+           fallthrough in-addr.arpa ip6.arpa
+        }
+        prometheus :9153
+        forward . /etc/resolv.conf
+        cache 30
+        loop
+        reload
+        loadbalance
+    }
+```
 
-Broadly based on [kubebuilder-declarative-pattern walkthrough](https://github.com/kubernetes-sigs/kubebuilder-declarative-pattern/blob/master/docs/addon/walkthrough/README.md)
+The above CR will install CoreDNS version `1.3.1`, with DNS Domain `cluster.local`, Service IP `10.96.0.10` and the Corefile defined in the CR.
 
-A few differences so we can use go modules and [crane](https://github.com/google/go-containerregistry/blob/master/cmd/crane/doc/crane.md) - neither of which are required, just personal preference.
+We can modify the specifications of CoreDNS by editing the Custom Resource.
 
-Created with kubebuilder:
+For example, we can upgrade the CoreDNS version to `1.6.7` here by editing the `version` spec in the CR to `1.6.7`. 
+This will enable the addon operator to install the manifests of CoreDNS associated with CoreDNS version `1.6.7`.
 
-```bash
-export KUBEBUILDER_ENABLE_PLUGINS=1
-kubebuilder init --fetch-deps=false --domain=x-k8s.io --license=apache2
+Another functionality that the operator provides while upgrading the CoreDNS version is the migration of the Corefile.
+The operator will check if the existing Corefile is compatible with the new version of CoreDNS (In this case, from 1.3.1 -> 1.6.7) and will make changes accordingly.
 
-kubebuilder create api --pattern=addon --controller=true --example=false --group=addons --kind=CoreDNS --make=false --namespaced=true --resource=true --version=v1alpha1
 
-```
+> NOTE: While it is possible to downgrade the CoreDNS version, it is NOT recommended.
 
-Run go mod vendor:
+Currently, the operator can be used by running it locally outside the cluster, or we can also run the operator in-cluster
 
-```bash
-go mod vendor
-```
+### Running the operator locally:
 
-Delete the test suites that are more checking that kubebuilder is working:
+We can register the CoreDNS CRD and a CoreDNS object, and then try running
+the controller locally.
+
+1) We need to generate and register the CRDs:
 
 ```bash
-find . -name "*_test.go" -delete
+$ make install
+  /Users/srajan/go/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
+  kustomize build config/crd | kubectl apply -f -
+  customresourcedefinition.apiextensions.k8s.io/coredns.addons.x-k8s.io created
 ```
 
-Commit
+To verify that the CRD has registered successfully:
 
 ```bash
-git add .
-git reset HEAD vendor
-git commit -m "Initial CoreDNS scaffolding"
+$ kubectl get crd coredns.addons.x-k8s.io
 ```
 
+2) Create a CoreDNS CR:
 
+```bash
+$ kubectl apply -f config/samples/addons_v1alpha1_coredns.yaml 
+coredns.addons.x-k8s.io/coredns-operator created
 
-Create the manifests (we bake them into the addon-operator by default):
+```
+
+To verify that the CR has been created successfully:
 
 ```bash
-mkdir -p channels/packages/coredns/1.3.1/
-pushd channels/packages/coredns/1.3.1/
-wget https://raw.githubusercontent.com/kubernetes/kubernetes/9b437f95207c04bf2f25ef3110fac9b356d1fa91/cluster/addons/dns/coredns/coredns.yaml.base
-cat coredns.yaml.base > manifest.yaml
-popd
+$ kubectl get coredns -n kube-system
+NAME               AGE
+coredns-operator   3m54s
 ```
 
-Define the stable channel:
+3) The controller can now be run using:
 
 ```bash
+make run
+```
 
-cat > channels/stable <<EOF
-manifests:
-- version: 1.3.1
-EOF
+We can see logs from the operator!
 
-```
+### Installing the operator in the cluster
 
-Running the operator locally:
+To start, build the operator image:
 
 ```bash
-make install
-make run
+make docker-build docker-push
 ```
-We can see logs from the operator!
 
+Once the image has been built successfully, to build the CRD and start the operator:
 
-Generally follow the [main instructions](https://github.com/kubernetes-sigs/kubebuilder-declarative-pattern/blob/master/docs/addon/walkthrough/README.md) at this point:
+```bash
+make deploy
+```
 
-* [enable the declarative pattern library in your types](https://github.com/kubernetes-sigs/kubebuilder-declarative-pattern/tree/master/docs/addon/walkthrough#adding-the-framework-into-our-types) and
-* [enable to declarative pattern in your controller](https://github.com/kubernetes-sigs/kubebuilder-declarative-pattern/tree/master/docs/addon/walkthrough#using-the-framework-in-the-controller)
-* finally add the [call to addon.Init](https://github.com/kubernetes-sigs/kubebuilder-declarative-pattern/tree/master/docs/addon/walkthrough#misc)
+You can troubleshoot the operator by inspecting the controller:
 
-Note that we intend to build these three steps into kubebuilder!
+```bash
+$ kubectl -n coredns-operator-system get deploy
+NAME                                  READY   UP-TO-DATE   AVAILABLE   AGE
+coredns-operator-controller-manager   1/1     1            1           111s
 
-Then follow the instructions for deploying onto kubernetes.
+# To check logs of the manager
+$ kubectl -n coredns-operator-system logs <coredns-operator-controller-manager-pod-name> manager
+```
diff --git a/walkthrough.md b/walkthrough.md
@@ -0,0 +1,197 @@
+# Writing an Addon Operator
+
+## What is it
+
+The [Addons via Operators KEP](kep) details how operators can be used for managing cluster addons. Below we will present a simple example, It should be straight-forward and give you an idea of how to proceed writing your own addon operator.
+
+## An example
+
+Bringing up CoreDNS in a Kubernetes cluster had been identified as a task that was clear and simple enough but still help us understand the general problem space and ask the right questions.
+You can review the code of the [CoreDNS addon operator](https://github.com/kubernetes-sigs/cluster-addons/tree/master/coredns) in this repository. 
+Here we will take you through the most interesting parts. [Its README](https://github.com/kubernetes-sigs/cluster-addons/tree/master/coredns/README.md) describes how the code scaffolding for the operator was set up, using `kubebuilder` and the [kubebuilder-declarative-pattern](https://github.com/kubernetes-sigs/kubebuilder-declarative-pattern).
+
+## Creating the operator
+
+Broadly based on [kubebuilder-declarative-pattern walkthrough](https://github.com/kubernetes-sigs/kubebuilder-declarative-pattern/blob/master/docs/addon/walkthrough/README.md)
+A few differences so we can use go modules and [crane](https://github.com/google/go-containerregistry/blob/master/cmd/crane/doc/crane.md) - neither of which are required, just personal preference.
+
+1. Created with kubebuilder:
+
+```bash
+export KUBEBUILDER_ENABLE_PLUGINS=1
+kubebuilder init --fetch-deps=false --domain=x-k8s.io --license=apache2
+
+kubebuilder create api --pattern=addon --controller=true --example=false --group=addons --kind=<my-addon> --make=false --namespaced=true --resource=true --version=v1alpha1
+
+```
+
+2. Run go mod vendor:
+
+```bash
+go mod vendor
+```
+
+3. Delete the test suites that are checking whether kubebuilder is working:
+
+```bash
+find . -name "*_test.go" -delete
+```
+
+4. Commit
+
+```bash
+git add .
+git reset HEAD vendor
+git commit -m "Initial addon scaffolding"
+```
+
+5. Create the manifests (we bake them into the addon-operator by default):
+
+```bash
+mkdir -p channels/packages/coredns/1.3.1/
+pushd channels/packages/coredns/1.3.1/
+wget https://raw.githubusercontent.com/kubernetes/kubernetes/9b437f95207c04bf2f25ef3110fac9b356d1fa91/cluster/addons/dns/coredns/coredns.yaml.base
+cat coredns.yaml.base > manifest.yaml
+popd
+```
+
+6. Define the stable channel:
+
+```bash
+
+cat > channels/stable <<EOF
+manifests:
+- version: 1.3.1
+EOF
+
+```
+
+7. Generally follow the [main instructions](https://github.com/kubernetes-sigs/kubebuilder-declarative-pattern/blob/master/docs/addon/walkthrough/README.md) at this point:
+
+* [enable the declarative pattern library in your types](https://github.com/kubernetes-sigs/kubebuilder-declarative-pattern/tree/master/docs/addon/walkthrough#adding-the-framework-into-our-types) and
+* [enable to declarative pattern in your controller](https://github.com/kubernetes-sigs/kubebuilder-declarative-pattern/tree/master/docs/addon/walkthrough#using-the-framework-in-the-controller)
+* finally add the [call to addon.Init](https://github.com/kubernetes-sigs/kubebuilder-declarative-pattern/tree/master/docs/addon/walkthrough#misc)
+
+Note that we intend to build these three steps into kubebuilder!
+
+Then follow the instructions for deploying onto kubernetes.
+
+7. Running the operator locally:
+
+```bash
+make install
+make run
+```
+We can see logs from the operator!
+
+### Breakdown structure of operator
+
+This is the structure of the operator after being created with Kubebuilder v2. The structure of any created operator will be similar to the one shown below
+This example is of the [CoreDNS addon operator](https://github.com/kubernetes-sigs/cluster-addons/tree/master/coredns)
+
+```sh
+.
+├── api
+│   ├── v1alpha1
+│   │   └── coredns
+│   │       ├── coredns_types.go
+│   │       ├── groupversion.go
+│   │       └── zz_generated.deepcopy.go
+├── channels
+│   ├── packages
+│   │   └── coredns
+│   │       ├── 1.3.1
+│   │       │   ├── clusterrole.yaml
+│   │       │   ├── clusterrolebinding.yaml
+│   │       │   ├── Corefile
+│   │       │   ├── deployment.yaml
+│   │       │   ├── kustomization.yaml
+│   │       │   ├── service.yaml
+│   │       │   └── serviceaccount.yaml
+│   │       ├── 1.6.7
+│   │       │   ├── clusterrole.yaml
+│   │       │   ├── clusterrolebinding.yaml
+│   │       │   ├── Corefile
+│   │       │   ├── deployment.yaml
+│   │       │   ├── kustomization.yaml
+│   │       │   ├── service.yaml
+│   │       │   └── serviceaccount.yaml
+│   │       └── 1.6.9
+│   │       │   ├── clusterrole.yaml
+│   │       │   ├── clusterrolebinding.yaml
+│   │       │   ├── Corefile
+│   │       │   ├── deployment.yaml
+│   │       │   ├── kustomization.yaml
+│   │       │   ├── service.yaml
+│   │       │   └── serviceaccount.yaml
+│   └── stable
+├── config
+│   ├── certmanager
+│   │   └── certificate.yaml
+│   │   └── kustomization.yaml
+│   │   └── kustomizeconfig.yaml
+│   ├── crds
+│   │   ├── bases
+│   │   │   └── addons.x-k8s.io_coredns.yaml
+│   │   ├── patches
+│   │   │   └── cainjection_in_coredns.yaml
+│   │   │   └── webhook_in_coredns.yaml
+│   │   ├── kustomization.yaml
+│   │   └── kustomizeconfig.yaml
+│   ├── default
+│   │   ├── kustomization.yaml
+│   │   ├── manager_auth_proxy_patch.yaml
+│   │   ├── manager_resource_patch.yaml
+│   │   ├── manager_webhook_patch.yaml
+│   │   └── webhookcainjection_patch.yaml
+│   ├── manager
+│   │   ├── kustomization.yaml
+│   │   └── manager.yaml
+│   ├── prometheus
+│   │   ├── kustomization.yaml
+│   │   └── monitor.yaml
+│   ├── rbac
+│   │   ├── auth_proxy_client_clusterrole.yaml
+│   │   ├── auth_proxy_role.yaml
+│   │   ├── auth_proxy_role_binding.yaml
+│   │   ├── auth_proxy_service.yaml
+│   │   ├── coredns_editor_role.yaml
+│   │   ├── coredns_viewer_role.yaml
+│   │   ├── kustomization.yaml
+│   │   ├── leader_election_role.yaml
+│   │   ├── leader_election_role_binding.yaml
+│   │   ├── role.yaml
+│   │   └── role_binding.yaml
+│   ├── samples
+│   │   └── addons_v1alpha1_coredns.yaml
+│   └── webhook
+│       ├── kustomization.yaml
+│       ├── kustomizeconfig.yaml
+│       └── manager.yaml
+├── controllers
+│   ├── tests
+│   │   ├── patches-stable.in.yaml
+│   │   ├── patches-stable.out.yaml
+│   │   ├── simple-stable.in.yaml
+│   │   └── simple-stable.out.yaml
+│   ├── coredns_controller.go
+│   ├── coredns_controller_test.go
+│   └── util.go
+├── hack
+│   ├── smoketest.go
+│   └── boilerplate.go.txt
+├── Dockerfile
+├── go.mod
+├── go.sum
+├── main.go
+├── Makefile
+├── PROJECT
+└── README.md
+```
+
+## Talk to us
+
+If you are interested in this, want to explore addon operators, want to discuss or get stuck somewhere, you can contact us at:
+
+- [#cluster-addons Slack](https://kubernetes.slack.com/messages/cluster-addons)
+- [SIG Cluster Lifecycle group](https://groups.google.com/forum/#!forum/kubernetes-sig-cluster-lifecycle)