Fix multiple KIC releases in helm

deployments/helm-chart/README.md

@@ -15,10 +15,26 @@ This chart deploys the NGINX Ingress controller in your Kubernetes cluster.
 
 ## Installing the Chart
 
+### Installing the CRDs
+
+By default, Helm installs a number of custom resource definitions (CRDs). Those CRDs are required for the VirtualServer, VirtualServerRoute and TransportServer custom resources.
+
+If you do not use those resources (which corresponds to `controller.enableCustomResources` set to `false`), you can skip the installation of the CRDs:
+
+* Using Helm 3.x client:
+
+    Specify `--skip-crds` for the helm install command.
+
+* Using a Helm 2.x client:
+
+    Set `controller.enableCustomResources` to `false`.
+
+> **Note**: If the CRDs are already installed in the cluster, Helm will skip the CRDs installation.
+
 ### Installing via Helm Repository
 
 1. Add NGINX Helm repository:
-    ```
+    ```console
     $ helm repo add nginx-edge https://helm.nginx.com/edge
     $ helm repo update
     ```
@@ -92,8 +108,38 @@ This chart deploys the NGINX Ingress controller in your Kubernetes cluster.
 
 > **Tip**: List all releases using `helm list`
 
+## Upgrading the Chart
+
+### Upgrading the CRDs
+
+Helm does not upgrade the CRDs during a release upgrade. Before you upgrade a release, run the following command to upgrade the CRDs:
+
+```console
+$ kubectl apply -f crds/
+```
+
+> **Note**: Make sure to check the [release notes](https://www.github.com/nginxinc/kubernetes-ingress/releases) for a new release for any special upgrade procedures.
+
+### Upgrading the Release
+
+To upgrade the release `my-release`
+
+* Using Helm 3.x client:
+
+    ```console
+    $ helm upgrade my-release
+    ```
+
+* Using Helm 2.x client:
+
+    ```console
+    $ helm upgrade -n my-release
+    ```
+
 ## Uninstalling the Chart
 
+### Uninstalling the Release
+
 To uninstall/delete the release `my-release`
 
 * Using Helm 3.x client:
@@ -110,6 +156,22 @@ To uninstall/delete the release `my-release`
 
 The command removes all the Kubernetes components associated with the chart and deletes the release.
 
+### Uninstalling the CRDs
+
+Uninstalling the release does not remove the CRDs. To remove the CRDs, run:
+
+```console
+$ kubectl delete crds transportservers.k8s.nginx.org virtualservers.k8s.nginx.org virtualserverroutes.k8s.nginx.org
+```
+
+> **Note**: The following command will delete all the corresponding custom resources in your cluster across all namespaces. Please ensure there are no custom resources that you want to keep and there are no other Ingress Controller releases running in the cluster.
+
+## Running Multiple Ingress Controllers
+
+If you are running multiple Ingress Controller releases in your cluster with enabled custom resources, the releases will share a single version of the CRDs. As a result, make sure that the Ingress Controller versions match the version of the CRDs. Additionally, when uninstalling a release, ensure that you don’t remove the CRDs until there are no other Ingress Controller releases running in the cluster.
+
+See [running multiple ingress controllers](https://docs.nginx.com/nginx-ingress-controller/installation/running-multiple-ingress-controllers/) for more details.
+
 ## Configuration
 
 The following tables lists the configurable parameters of the NGINX Ingress controller chart and their default values.

deployments/helm-chart/templates/controller-ts-definition.yaml → deployments/helm-chart/crds/transportserver.yaml

@@ -1,10 +1,11 @@
-{{- if .Values.controller.enableCustomResources }}
 apiVersion: apiextensions.k8s.io/v1beta1
 kind: CustomResourceDefinition
 metadata:
   name: transportservers.k8s.nginx.org
   labels:
-    {{- include "nginx-ingress.labels" . | nindent 4 }}
+    app.kubernetes.io/name: "nginx-ingress"
+  annotations:
+    "helm.sh/hook": crd-install
 spec:
   group: k8s.nginx.org
   versions:
@@ -76,4 +77,3 @@ spec:
                     type: integer
                   service:
                     type: string
-{{- end }}

deployments/helm-chart/templates/controller-vs-definition.yaml → deployments/helm-chart/crds/virtualserver.yaml

@@ -1,10 +1,11 @@
-{{- if .Values.controller.enableCustomResources }}
 apiVersion: apiextensions.k8s.io/v1beta1
 kind: CustomResourceDefinition
 metadata:
   name: virtualservers.k8s.nginx.org
   labels:
-    {{- include "nginx-ingress.labels" . | nindent 4 }}
+    app.kubernetes.io/name: "nginx-ingress"
+  annotations:
+    "helm.sh/hook": crd-install
 spec:
   group: k8s.nginx.org
   versions:
@@ -648,4 +649,3 @@ spec:
               type: string
             state:
               type: string
-{{- end }}

deployments/helm-chart/templates/controller-vsr-definition.yaml → deployments/helm-chart/crds/virtualserverroute.yaml

@@ -1,10 +1,11 @@
-{{- if .Values.controller.enableCustomResources }}
 apiVersion: apiextensions.k8s.io/v1beta1
 kind: CustomResourceDefinition
 metadata:
   name: virtualserverroutes.k8s.nginx.org
   labels:
-    {{- include "nginx-ingress.labels" . | nindent 4 }}
+    app.kubernetes.io/name: "nginx-ingress"
+  annotations:
+    "helm.sh/hook": crd-install
 spec:
   group: k8s.nginx.org
   versions:
@@ -632,4 +633,3 @@ spec:
               type: string
             state:
               type: string
-{{- end }}

deployments/helm-chart/templates/crds.yaml

@@ -0,0 +1,7 @@
+{{- if .Values.controller.enableCustomResources }}
+{{- range $path, $bytes := .Files.Glob "crds/*.yaml" }}
+{{ $.Files.Get $path }}
+---
+{{- end }}
+{{- end }}
+