Proposal for "CNOE Supported" Packages

[blakeromano]

We want to be able to have a way to have a core set of components that CNOE supports and that tools like IDPBuilder have an easy way and schema to add supported packages while being able to make "non-supported" changes. As a platform engineer we should have a way to quickly add components to IDPBuilder or our GitOps repo that follow a "CNOE" standard on application/applicationset definition. Ideally IDPBuilder becomes a plug and play model where testing components from your actual GitOps in a local environment.

Notes to consider surrounding this proposal:

1. We need to define a specification for what CNOE expects "plug and play" packages to look like. We may want to consider moving towards applicationsets rather then applications in ArgoCD.
2. We can look at the patterns in https://github.com/gitops-bridge-dev to determine how we want to define that specification.
3. We do not want to become a package manager or worry about dealing with dependencies at scale. We should consider versioning addons with some sort of release process and only guarantee that the versions specified in that release work with each other. We do want to allow developers to override or configure additional resources but CNOE will not support anything broken with the difference in versions from the "standard".
4. We will want to have a way for developers to easily add "CNOE Supported" packages. We will want a static repository for those packages. We may want to consider a static public Github repository for that like https://github.com/gitops-bridge-dev/gitops-bridge-argocd-control-plane-template where we can grab the packages from Github directly, deal with tags/releases to track standard package versions, etc;.
5. Standard Packages should be omitted from any binaries or not become "required" to run IDPBuilder or CNOE. We want to keep the necessary components limited to tools like ArgoCD, Ingress NGINX, and Gitea (for local IDPBuilder)

[csantanapr]

We can look at the patterns in https://github.com/gitops-bridge-dev to determine how we want to define that specification.

One benefit of using argocd application sets is that you can start with a set of components at a certain version deployed on the first cluster this can be the kind cluster with idpbuilder, but if you wan to deploy the same or subset of components to another cluster lets say a managed cluster like EKS its a matter of using the same application sets definition on another cluster even having different versions or using progressive syncs to promote a newer version from one environment cluster to the next.
This allows the platform engineer to manage the component across clusters from a single source of truth, using hub-spoke or each cluster in stand-alone using the same application set applied to it-self