App of Apps not working when using kustomize applications #3181
Comments
|
Sorry for commenting on this issue, but I seem to be running into the same using helm app of apps too. Running a sync results in the following And looking at the the Whereas adding the applications individually via the UI seems to work without any issues. argo version |
|
Same problem as above with argo 1.5.4 and kustomize templates. However, mine says sync is successful, however no application pods are created. Works fine if i create app directly. I have a full repository available if someone can double check: After more experiements, I was able to fix the issue by adding namespace under yaml definition in the apps folder It has to have argocd namespace, anything else and it doesn't work. In my deployment I add everything into "demo" namespace and argocd runs under default argocd namespace. |
|
I have the same error w/ a Helm App of Apps. All application manifests specify edit: for anyone who found themselves here while trying out the App of App patterns. If your child apps are deploying to a different cluster, make sure your parent app has destination |
|
This is still a problem in 1.6.1 The error from the application controller: |
|
One thing I've discovered in my research is the API endpoint I'm wondering, is it that ArgoCD isn't recognizing Kustomize is involved so trying to actually add the Kustomization type to the cluster rather than processing it? If so, how does ArgoCD get triggered on this? In the interest of discovering that information, but not being a Golang guy, I found these hopefully relevant bits. There is apparently two ways this is supposed to be triggered. There is some documentation in the Tool Detection Doc, but it left me with some questions and clearly it isn't working exactly as documented. First, auto-detection based on the contents of the repo. So what does the code use to determine if a directory of code should be handled as a Kustomize appType? This is then used on line 181 in the isKustomization function. It appears very consistent with the docs. This info is used by Second, explicit identification in the The two methods of detection appear to come together in My suspicion is that we aren't making it this far, but I don't know how to watch the code at work. |
|
I think I have finally found the trigger point. All Failing scenario: kind: Application
spec:
source:
directory:
recurse: trueSuccessful scenario: kind: Application
spec:
source:
directory:
recurse: false |
This works for app of apps where all of the children are apps. If any of the children are simple resources (e.g. Secret), the end up being created in the local cluster and not the external one. |
|
is this really still an issue? I'm POC'ing ArgoCD now but this bug makes it seem like App of Apps configuration is broken and so what's the point |
|
I'm in the same boat as @bitva77 . Our app of apps is a mixture of applications (off the shelf helm charts) and template files in the app of apps chart. Is that not supported? |
|
It does seem to work, but there are some gotchas along the way. The one that had us really twisted is that you can NOT set up the app to be recursive or it doesn't properly detect what is going on and ArgoCD gets terribly confused. |
I tried this, and the app stayed out of sync. It could never sync the directory recurse section. Apparently recurse: false is the default behavior. I ended up not using this tiny kustomize file, instead copying in the entire 26,000 line cert-manager kube file. Bummer, but it works. |
Is there any way around this? I'm specifically trying to use a number of kustomize.yml configs in my repo and rely on all applications being recursively deployed from a kustomize file but I'm seeing the same sync issues unless I disable recursion In my case I'm trying to run a bootstrap that then adds an app-of-apps to argocd where I expect my source of truth for everything to be. .
├── argocd
│ ├── argocd
│ │ └── kustomization.yml
│ └── sealed-secrets
│ └── kustomization.yml
├── bootstrap
│ └── kustomization.yml
├── juju
│ ├── deployment.yml
│ └── README.md
├── kustomize
│ ├── argocd
│ │ ├── app.yml
│ │ ├── cm.rbac.yml
│ │ ├── cm.repos.yml
│ │ ├── cm.sso.yml
│ │ ├── kustomization.yml
│ │ ├── overlay.argocd.yml
│ │ ├── overlay.secrets.yml
│ │ ├── resource.ingress.yml
│ │ ├── resource.namespace.yml
│ │ ├── resource.project.yml
│ │ └── resource.secrets.yml
│ └── sealed-secrets
│ ├── app.yml
│ └── kustomization.yml
└── README.md# argocd/argocd/kustomization.yml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../kustomize/argocd/# kustomize/argocd/app.yml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: argocd
namespace: argocd
spec:
source:
path: argocd
directory:
recurse: true
... |
|
Hi guys, any news about this issue? |
|
That issue still exist I m using 2.0.5 and in the middle of the POC with kustomization app of apps repo. I already tried @Woland2k and @FearTheBadger recommendation but still have no luck child apps stuck at OutOfSync argocd: v2.0.5+4c94d88.dirty |
|
I can confirm. This issue still exists to this date: What it does, ArgoCD recognised generated resource (in my case Deployment), but ended with "Sync Failed" and message: "the server could not find the requested resource" |
|
I suppose it makes sense that this wouldn't work, since there is ambiguity in how to handle a directory containing both an It would be nice if Argo was "smarter" about how to deploy the app of apps when using Kustomize, i.e., allow recursive directories, but only for spec:
directory:
recurse: true
pattern: **/app.yamlFor the time being, my seed app is set to |
|
I'm facing the same issue and can't figure out why it isn't working. I'm sure it has worked at some point. See here my app of apps. apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: opa-policies
namespace: argocd
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
destination:
namespace: opa-policies
server: {{ .Values.spec.destination.server }}
project: my-project
source:
path: opa-policies
repoURL: {{ .Values.spec.source.repoURL }}
targetRevision: {{ .Values.spec.source.targetRevision }}
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=trueThen in the folder opa-policies I have the following Kustomize. apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
# namespace: gatekeeper-system
resources:
# - https://github.com/sigstore/cosign-gatekeeper-provider/raw/main/manifest/deployment.yaml
# - https://github.com/sigstore/cosign-gatekeeper-provider/raw/main/manifest/service.yaml
# - https://github.com/sigstore/cosign-gatekeeper-provider/raw/main/manifest/provider.yaml
- templates/container-resources-template.yaml
- templates/image-tagging-template.yaml
- templates/cosign-template.yaml
- constraints/image-must-not-have-latest-tag-constraint.yaml
- constraints/image-must-have-limits-and-requests-constraint.yaml
- constraints/image-signed-constraint.yaml
# patches:
# - target:
# kind: Provider
# group: externaldata.gatekeeper.sh
# version: v1alpha1
# name: cosign-gatekeeper-provider
# patch: |-
# - op: add
# path: /metadata/annotations/argocd.argoproj.io~1sync-wave
# value: -2
# - op: add
# path: /spec/insecureTLSSkipVerify
# value: true
# - op: replace
# path: /spec/timeout
# value: 15
# - op: replace
# path: /spec/url
# value: http://cosign-gatekeeper-provider.gatekeeper-system:8090/validate
# - target:
# kind: Deployment
# group: apps
# version: v1
# name: cosign-gatekeeper-provider
# patch: |-
# - op: add
# path: /metadata/annotations/argocd.argoproj.io~1sync-wave
# value: -2
# - target:
# kind: Service
# version: v1
# name: cosign-gatekeeper-provider
# patch: |-
# - op: add
# path: /metadata/annotations/argocd.argoproj.io~1sync-wave
# value: -2The commented lines I have tested with to figure out if it is related to any kustomize specifics in combination with ArgoCD. The result is that ArgoCD thinks all is synchronized, however the App is not created. time="2022-08-16T09:13:35Z" level=info msg="Applying resource Config/config in cluster: https://172.20.0.1:443, namespace: gatekeeper-system"
time="2022-08-16T09:13:35Z" level=info msg="Adding resource result, status: 'Synced', phase: 'Running', message: 'config.config.gatekeeper.sh/config unchanged'" application=my-policies kind=Config name=config namespace=gatekeeper-system phase=Sync syncId=00052-MtTBa
time="2022-08-16T09:13:35Z" level=info msg="Applying resource Application/opa-policies in cluster: https://172.20.0.1:443, namespace: argocd"
time="2022-08-16T09:13:35Z" level=info msg="Adding resource result, status: 'Synced', phase: 'Running', message: 'application.argoproj.io/opa-policies unchanged'" application=my-policies kind=Application name=opa-policies namespace=argocd phase=Sync syncId=00052-MtTBaSee this screenshot all is synced according to ArgoCD, but when clicking the red circled icon the app doesn't actually exist, neither it is applied.
See here the screenshot showing the app that pretends to by synced actually isn't.
Strange thing is there are no errors, and nothing in the logs. |
|
Thanks to @crenshaw-dev I was able to resolve the issue. Solution All applications have to be installed in the argocd namespace. So like apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: policies
namespace: argocd
finalizers:
- resources-finalizer.argocd.argoproj.io
…Following is not yet supported and will probably land in the next apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: policies
namespace: my-other-namespace
finalizers:
- resources-finalizer.argocd.argoproj.io
…So |
I am getting an error mentioned by this ticket, when I tried to use a yaml with The error that I got: |
|
+1 |
|
This should be supported since The key points here are that the Along with that, the additional RBAC resources must be created to grant argo the access it needs. This is handled in the other source that I had outlined. After updating the config & RBAC, you can now create argo Applications in new namespaces. When creating the After adding this, I can deploy the argo application to a separate namespace, and I have been able to successfully deploy a combination of helm resources & manifests to the the same destination namespace from a single app of apps. |
|
Closing because I think most of the issues here are due to a misunderstanding of how an app-of-apps works. An app-of-apps isn't really anything special. It's just an Application CR that deploys the manifests that happen to be other Applications. Applications must live on the same cluster as the Argo CD app controller, either in the same namespace as the app controller, or in an explicitly-configured additional namespace using the "apps in any namespace" feature. In other words, the The resources managed by the child Applications could theoretically be any kind and could be destined for any namespace on any cluster. If you have a directory structure like this: then you need to make sure your app-of-apps' child Applications have recursion turned off. Otherwise the parent app is going to pick up configmap.yaml and deployment.yaml and try to deploy them to the Argo CD cluster in the app controller's namespace (at least, if your parent app is configured correctly). If you have a directory structure like this: then disabling recursion is not enough -- you need to configure the parent app to ignore everything besides the application.yaml files. Important: I don't recommend keeping your Application manifests in the same repo as other resources, unless the people writing/reviewing/merging those resources are Argo CD admins or anyway or unless you're using the Apps-in-any-namespace pattern with a carefully-crafted parent app. If a random user can set the Hope that's clear. Closing since I think this is a collection of misunderstandings of how Apps-of-apps work. Feel free to continue to discuss though! |
and others
Checklist:
argocd version.Describe the bug
When creating an Apps of Apps, where each app is a kustomize app, apps donot get created. They throw the following error.
one or more synchronization tasks are not valid. If i apply manually the app alone it works.To Reproduce
Create a kustomize app which points to another kustomize app.
Expected behavior
Apps of Apps should be created.
Screenshots
Version
argocd: v1.4.2+48cced9
BuildDate: 2020-01-24T01:07:43Z
GitCommit: 48cced9
GitTreeState: clean
GoVersion: go1.12.6
Compiler: gc
Platform: darwin/amd64
argocd-server: v1.4.2+48cced9
BuildDate: 2020-01-24T01:07:03Z
GitCommit: 48cced9
GitTreeState: clean
GoVersion: go1.12.6
Compiler: gc
Platform: linux/amd64
Ksonnet Version: v0.13.1
Kustomize Version: Version: {Version:kustomize/v3.2.1 GitCommit:d89b448c745937f0cf1936162f26a5aac688f840 BuildDate:2019-09-27T00:10:52Z GoOs:linux GoArch:amd64}
Helm Version: v2.15.2
Kubectl Version: v1.14.0
The text was updated successfully, but these errors were encountered: