The NLB created by istio-ingress service does not have cross-zone loadbalancing enabled as desired

[vchintal]

Description

In the istio pattern, the service for the istio-ingress (Helm Chart) is provided desired annotation on this line which enables the cross-zone load balancing on the NLB created as a result. However, the processing of such AWS specific annotations is contingent upon the AWS Loadbalancer Controller being deployed and ready prior to any service of Loadbalancer type deployed in the cluster.

The way the Terraform code is structured in main.tf the order of creation of AWS Loadbalancer Controller and the istio-ingress Helm chart deployment is not deterministic. And because of that

1. The AWS Loadbalancer Controller is deployed after the istio-ingress service deployment and
2. The annotations provided on the istio-ingress service is not picked up, processed and hence the cross-zone loadbalancing is not enabled as desired.

Reproduction Code [Required]

Steps to reproduce the behavior:

1. Follow the deployment steps in the istio pattern and if possible capture the deployment order of AWS Loadbalancer Controller and istio-ingress Helm chart
2. Go to the EC2 Console → Load Balancers → Choose the one created by istio-ingress service → click on Attributes tab and verify if the cross-zone loadbalancing is enabled

Expected behaviour

Cross-zone loadbalancing is enabled

Actual behaviour

Cross-zone loadbalancing is disabled

Terminal Output Screenshot(s)

[vchintal]

• @bryantbiggs

[bryantbiggs]

since we have multi-language blueprints, how would we solve this for alternative deployment methods? Meaning - if I use a GitOps controller, how do I resolve this issue?

[vchintal]

I do not have a personal recommendation or suggestion here but I would like to ask how other patterns are dealing with such a requirement?

Setting this specific usecase aside, it not uncommon for an operator/controller (with CRDs) and a dependent application (with CRs) to be both packaged as Helm charts. How will we (or plan to) handle cases when they both need to deployed in a specific order?

I reached out to my ex-Red Hat colleague Christian Hernandez who is the author of this post https://codefresh.io/blog/argo-cd-application-dependencies/ and he says "App of Apps pattern is something most people do" and he offered me sequence of steps for ArgoCD:

1. App of Apps + Sync waves
2. Sync waves within each app
3. enable retries for each app (because the first time trying to apply something will probably fail)
4. I do SkipDryRunOnMissingResource=true annotation anytime I'm doing anything that needs a CRD. This annotation (along with the retries) will make Argo CD keep trying to apply the manifests until it's successful