Source is not ready, artifact not found #1545

dzmitry-lahoda · 2021-06-18T19:47:20Z

Describe the bug

Referencing private repository does not not work. I recreated report and reinstalled flux several times. Interesting, that repo were flux is located is synced with flux well, while repo under same org and token does not work. Workaround is to host flux-system and other kustomization in same private repo.

Reconciliation gives errors

Source is not ready, artifact not found

flux reconcile kustomization hangs for that repo.

To Reproduce

Steps to reproduce the behaviour:

Follow latest up to date installation instruction
I cannot provide private repo...

Expected behavior

Better error description.

Additional context

Kubernetes version: 18.19
Git provider: github
Container registry provider: ACR

Below please provide the output of the following commands:


► checking prerequisites
✔ kubectl 1.20.1 >=1.18.0-0
✔ Kubernetes 1.18.19 >=1.16.0-0
► checking controllers
✔ helm-controller: deployment ready
► ghcr.io/fluxcd/helm-controller:v0.11.1
✔ image-automation-controller: deployment ready
► ghcr.io/fluxcd/image-automation-controller:v0.12.0
✔ image-reflector-controller: deployment ready
► ghcr.io/fluxcd/image-reflector-controller:v0.10.0
✔ kustomize-controller: deployment ready
► ghcr.io/fluxcd/kustomize-controller:v0.13.0
✔ notification-controller: deployment ready
► ghcr.io/fluxcd/notification-controller:v0.15.0
✔ source-controller: deployment ready
► ghcr.io/fluxcd/source-controller:v0.15.1
✔ all checks passed

NAME                                               READY   STATUS    RESTARTS   AGE
pod/helm-controller-b5c45677f-rgff9                1/1     Running   0          2m7s
pod/image-automation-controller-685966b855-7hmrs   1/1     Running   0          2m7s
pod/image-reflector-controller-5ffb9c67c8-b6gt8    1/1     Running   0          2m6s
pod/kustomize-controller-798c8dbdb9-rzq9g          1/1     Running   0          2m6s
pod/notification-controller-66545867f4-gnn9d       1/1     Running   0          2m6s
pod/source-controller-78945f55fb-krf2z             1/1     Running   0          2m6s

NAME                              TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
service/notification-controller   ClusterIP   10.0.170.111   <none>        80/TCP    2m8s
service/source-controller         ClusterIP   10.0.122.194   <none>        80/TCP    2m7s
service/webhook-receiver          ClusterIP   10.0.169.70    <none>        80/TCP    2m7s

NAME                                          READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/helm-controller               1/1     1            1           2m7s
deployment.apps/image-automation-controller   1/1     1            1           2m7s
deployment.apps/image-reflector-controller    1/1     1            1           2m6s
deployment.apps/kustomize-controller          1/1     1            1           2m6s
deployment.apps/notification-controller       1/1     1            1           2m6s
deployment.apps/source-controller             1/1     1            1           2m6s

NAME                                                     DESIRED   CURRENT   READY   AGE
replicaset.apps/helm-controller-b5c45677f                1         1         1       2m7s
replicaset.apps/image-automation-controller-685966b855   1         1         1       2m7s
replicaset.apps/image-reflector-controller-5ffb9c67c8    1         1         1       2m6s
replicaset.apps/kustomize-controller-798c8dbdb9          1         1         1       2m6s
replicaset.apps/notification-controller-66545867f4       1         1         1       2m6s
replicaset.apps/source-controller-78945f55fb             1         1         1       2m6s

The text was updated successfully, but these errors were encountered:

stefanprodan · 2021-06-19T05:42:38Z

Please post here:

flux get sources all --all-namespaces
flux logs --level=error

dzmitry-lahoda · 2021-06-19T13:18:27Z

I do not have access to these errors, but thans for commands, helped a lot with other issue.

I may be related git submodules (or upgrade from 14 to 15 - i pretty clear sure recalled that on 14 it worked for me once, than in attempt to fix images, update to 15). Not sure. I have not merged all code into 1 repo. Will recheck the issue when will upgrade to newer kustomize when we will deploy production cluster within month.

For now I have merged all stuff into one repo.

matheusrazzup · 2021-06-29T21:48:40Z

Im facing the same here, any updates about this?

hiddeco · 2021-06-29T22:03:36Z

Without providing any further information about the condition your GitRepository is in, and what it reports using e.g. flux get sources all --all namespaces or kubectl describe gitrepository <name>.

It is hard, if not impossible for us, to further help with debugging.

matheusrazzup · 2021-06-29T22:32:42Z

flux get sources all --all-namespaces:

And error logs as suggested by stefan (flux logs --level=error)

Apparently its not able to clone the repo via ssh, maybe im missing some param in bootstrap?

PS-1: I just followed the "Get Started" guide present in the docs
PS-2: Im using kind

hiddeco · 2021-06-30T07:50:37Z

Can you provide the value of the known_hosts entry in the flux-system secret (which does not contain any sensitive data), and just to be sure, the output of executing getent hosts github.com in the source-controller container, and on the machine you are running flux from?

knopka · 2021-06-30T09:40:45Z

Faced with similar error following by Getting Started guide

$ flux get kustomizations
NAME       	READY	MESSAGE                                                        	REVISION                                     	SUSPENDED
flux-system	True 	Applied revision: main/bbea8280f1b7582c0874cbfad25d5cf93717aa7d	main/bbea8280f1b7582c0874cbfad25d5cf93717aa7d	False
podinfo    	False	Source is not ready, artifact not found                        	                                             	False
$
$
$ flux get sources all --all-namespaces
NAMESPACE  	NAME                     	READY	MESSAGE                                                                                                                                      	REVISION                                     	SUSPENDED
flux-system	gitrepository/datadog    	False	unable to clone 'ssh://[email protected]/****/flux-22', error: error creating SSH agent: "SSH agent requested but SSH_AUTH_SOCK not-specified"	                                             	False
flux-system	gitrepository/flux-system	True 	Fetched revision: main/bbea8280f1b7582c0874cbfad25d5cf93717aa7d                                                                              	main/bbea8280f1b7582c0874cbfad25d5cf93717aa7d	False

$

hiddeco · 2021-06-30T09:48:48Z

Can you provide the YAML for gitrepository/datadog and kustomization/podinfo?

knopka · 2021-06-30T09:52:12Z

Can you provide the YAML for gitrepository/datadog and kustomization/podinfo?

apiVersion: source.toolkit.fluxcd.io/v1beta1
kind: GitRepository
metadata:
  name: datadog
  namespace: flux-system
spec:
  interval: 30s
  ref:
    branch: main
  url: ssh://[email protected]/knopka/flux-22

apiVersion: kustomize.toolkit.fluxcd.io/v1beta1
kind: Kustomization
metadata:
  name: podinfo
  namespace: flux-system
spec:
  interval: 1m0s
  path: ./apps
  prune: true
  sourceRef:
    kind: GitRepository
    name: datadog
  validation: client

hiddeco · 2021-06-30T09:54:06Z

Your GitRepository is missing a secret reference, and without a private key, cloning over SSH is impossible. You either need to change it to an HTTPS URL (but if the repository is private, you would still need an authentication token in a secret, and a reference to it), or reference to the secret that contains a private key which has access to the repository.

knopka · 2021-06-30T10:10:03Z

@hiddeco Thank you! You were right, after adding secret reference all works. This is described in https://fluxcd.io/docs/components/source/gitrepositories/#ssh-authentication

dzmitry-lahoda · 2021-06-30T12:40:52Z

in my case i had secret. it worked, before i have upgraded. but anyway need to recheck. More direct error output from ssh could be welcome.

matheusrazzup · 2021-06-30T18:13:12Z

Can you provide the value of the known_hosts entry in the flux-system secret (which does not contain any sensitive data), and just to be sure, the output of executing getent hosts github.com in the source-controller container, and on the machine you are running flux from?

Here is the ouptput:

➜  ~ kg secret flux-system -n flux -o jsonpath={.data.known_hosts} | base64 -d
github.com ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAq2A7hRGmdnm9tUDbO9IDSwBK6TbQa+PXYPCPy6rbTrTtw7PHkccKrpp0yVhp5HdEIcKr6pLlVDBfOLX9QUsyCOV0wzfjIJNlGEYsdlLJizHhbn2mUjvSAHQqZETYP81eFzLQNnPHt4EVVUh7VfDESU84KezmD5QlWpXLmvU31/yMf+Se8xhHTvKSCZIFImWwoG6mbUoWf9nzpIoaSjB+weqqUUmpaaasXVal72J+UX2B+2RPW3RcT0eOzQgqlJL3RKrTJvdsjE3JEAvGq3lGHSZXy28G3skua2SmVi/w4yCE6gbODqnTWlg7+wC604ydGXA8VJiS5ap43JXiUFFAaQ==%              
➜  ~ kex source-controller-8c56bb89f-9stgm bash -n flux
controller@source-controller-8c56bb89f-9stgm:/$ getent hosts github.com 
208.113.218.221 github.aaakk.us.kg.br
controller@source-controller-8c56bb89f-9stgm:/$ exit
exit
➜  ~ getent hosts github.com 
140.82.113.3    github.com

Noticed that getent hosts github.com is different within source-controller container and my machine

PS: I did bootstrap from my machine (where getent displayed 140.82.113.3 github.com)

UPDATE: It worked now! I had a search for "br" in my resolv.conf, since kind extends my resolution it was completing with that. I really appreciate your support! @hiddeco

kingdonb · 2021-07-08T15:26:17Z

It looks like this issue has been resolved! We can open a new issue if there was some documentation that we should add.

This was referenced Jun 18, 2021

Image Automation Does not work in specific case (if there is submodule) #1546

Closed

Azure DevOps: SSH Unable to extract public key from private key (after upgrade from flux 0.13.3 to 0.15.1) #1543

Closed

kingdonb closed this as completed Jul 8, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Source is not ready, artifact not found #1545

Source is not ready, artifact not found #1545

dzmitry-lahoda commented Jun 18, 2021 •

edited

Loading

stefanprodan commented Jun 19, 2021

dzmitry-lahoda commented Jun 19, 2021 •

edited

Loading

matheusrazzup commented Jun 29, 2021

hiddeco commented Jun 29, 2021

matheusrazzup commented Jun 29, 2021

hiddeco commented Jun 30, 2021 •

edited

Loading

knopka commented Jun 30, 2021

hiddeco commented Jun 30, 2021

knopka commented Jun 30, 2021

hiddeco commented Jun 30, 2021

knopka commented Jun 30, 2021

dzmitry-lahoda commented Jun 30, 2021

matheusrazzup commented Jun 30, 2021 •

edited

Loading

kingdonb commented Jul 8, 2021

Source is not ready, artifact not found #1545

Source is not ready, artifact not found #1545

Comments

dzmitry-lahoda commented Jun 18, 2021 • edited Loading

Describe the bug

To Reproduce

Expected behavior

Additional context

stefanprodan commented Jun 19, 2021

dzmitry-lahoda commented Jun 19, 2021 • edited Loading

matheusrazzup commented Jun 29, 2021

hiddeco commented Jun 29, 2021

matheusrazzup commented Jun 29, 2021

hiddeco commented Jun 30, 2021 • edited Loading

knopka commented Jun 30, 2021

hiddeco commented Jun 30, 2021

knopka commented Jun 30, 2021

hiddeco commented Jun 30, 2021

knopka commented Jun 30, 2021

dzmitry-lahoda commented Jun 30, 2021

matheusrazzup commented Jun 30, 2021 • edited Loading

kingdonb commented Jul 8, 2021

dzmitry-lahoda commented Jun 18, 2021 •

edited

Loading

dzmitry-lahoda commented Jun 19, 2021 •

edited

Loading

hiddeco commented Jun 30, 2021 •

edited

Loading

matheusrazzup commented Jun 30, 2021 •

edited

Loading