Kamelet http-sink not working

[christophd]

The http-sink Kamelet raises errors during trait customization such as:

component not found for uri \"{{url}}\" in camel catalog runtime version 1.16.0

Error details

{"level":"error","ts":1670923780.0970173,"logger":"controller.integration-controller","msg":"Reconciler error","reconciler group":"camel.apache.org","reconciler kind":"Integration","name":"aws-s3-to-http","namespace":"yaks-demo","error":"error during trait customization: component not found for uri \"{{url}}\" in camel catalog runtime version 1.16.0","errorVerbose":"component not found for uri \"{{url}}\" in camel catalog runtime version 1.16.0\nerror during trait customization\ngithub.com/apache/camel-k/pkg/trait.Apply\n\tgithub.com/apache/camel-k/pkg/trait/trait.go:59\ngithub.com/apache/camel-k/pkg/controller/integration.(*initializeAction).Handle\n\tgithub.com/apache/camel-k/pkg/controller/integration/initialize.go:54\ngithub.com/apache/camel-k/pkg/controller/integration.(*reconcileIntegration).Reconcile\n\tgithub.com/apache/camel-k/pkg/controller/integration/integration_controller.go:349\ngithub.com/apache/camel-k/pkg/util/monitoring.(*instrumentedReconciler).Reconcile\n\tgithub.com/apache/camel-k/pkg/util/monitoring/controller.go:70\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227\nruntime.goexit\n\truntime/asm_arm64.s:1172","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}

The Kamelet uses a to: {{url}} where url is specified as Kamelet property (see https://github.com/apache/camel-kamelets/blob/main/kamelets/http-sink.kamelet.yaml#L61)

This has been working with prior Camel K versions. I have encountered this with with v1.11.0

[christophd]

@tadayosi could you please have a look. I think there were some changes related to non resolved components/dependencies during inspection

[mwmahlberg]

Same is true at least for the http-secured-source.

[oscerd]

Same is true at least for the http-secured-source.

Commenting on a closed PR won't help. What is version? What is the error? What is not working? Open another issue and report the problem. A comment won't help reproducing the problem. Also the Kamelets are hosted at https://github.com/apache/camel-kamelets/tree/main/kamelets

[mwmahlberg]

@oscerd Working on it ;)

There is still something not quite working out. I run into the following errors for the http-secured-source with camel-k version s 1.12.10, 2.0.1 and 2.1.0. Below you will find the log for Camel K Client 2.1.0 & Camel K Operator 2.1.0, but it looked pretty much the same for all three versions

{"level":"error","ts":"2023-11-10T17:37:21Z","msg":"Reconciler error","controller":"integration-controller","controllerGroup":"camel.apache.org","controllerKind":"Integration","Integration":{"name":"http-secured-source-to-minio-sink","namespace":"test"},"namespace":"test","name":"http-secured-source-to-minio-sink","reconcileID":"6ea19c74-dfa3-43a8-aa5a-08a44932abb3","error":"error during trait customization: dependencies trait execution failed: component not found for uri \"{{url}}?authMethod=%7B%7BauthMethod%7D%7D&authPassword=%7B%7BauthPassword%7D%7D&authUsername=%7B%7BauthUsername%7D%7D&authenticationPreemptive=%7B%7BauthenticationPreemptive%7D%7D\" in camel catalog runtime version 3.2.0","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:324\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:265\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:226"}

if we look at the url-decoded uri string, it seems to fit:

{{url}}?authMethod={{authMethod}}&authPassword={{authPassword}}&authUsername={{authUsername}}&authenticationPreemptive={{authenticationPreemptive}}

The Kamelet-Bindings or in this version the Pipe is rather simple:

apiVersion: camel.apache.org/v1
kind: Pipe
metadata:
  annotations:
    camel.apache.org/operator.id: camel-k
  creationTimestamp: null
  name: http-secured-source-to-minio-sink
  namespace: test
spec:
  sink:
    properties:
      accessKey: <redacted>
      autoCreateBucket: "true"
      bucketName: customer
      endpoint: http://test-hl.test.svc.cluster.local:9000
      secretKey: <redacted>
    ref:
      apiVersion: camel.apache.org/v1
      kind: Kamelet
      name: minio-sink
      namespace: test
  source:
    properties:
      authMethod: Basic
      authPassword: <redacted>
      authUsername: <redacted>
      url: http://demo.<redacted>.sslip.io
    ref:
      apiVersion: camel.apache.org/v1
      kind: Kamelet
      name: http-secured-source
      namespace: test
status: {}

One of my assumptions was that this was a problem with eager loading and the sslip.io address, which turned out to be wrong: The address resolves within the cluster and I did change it to the according IP.

Now one thing I noticed is that in the error string, the {{ and }} are url encoded for the query string, but not for the url:

"{{url}}?authMethod=%7B%7BauthMethod%7D%7D&authPassword=%7B%7BauthPassword%7D%7D&authUsername=%7B%7BauthUsername%7D%7D&authenticationPreemptive=%7B%7BauthenticationPreemptive%7D%7D"

I'd assume that the problem is somewhere located about this... incsonsistency, because sure enough, IsResolvable fails:
https://go.dev/play/p/C5kPnCliqmw

---

Some additional version information (redacted for brevity):

Catalog

apiVersion: camel.apache.org/v1
kind: CamelCatalog
metadata:
  annotations:
    camel.apache.org/operator.id: camel-k
  generation: 1
  labels:
    app: camel-k
    camel.apache.org/catalog.loader.version: 4.0.0
    camel.apache.org/catalog.version: 4.0.0
    camel.apache.org/runtime.version: 3.2.0
...

Knative

$ kn version
Version:      v1.9.0
Build Date:   2023-01-26 20:34:03
Git Revision: df40f5a3
Supported APIs:
* Serving
  - serving.knative.dev/v1 (knative-serving v1.9.0)
* Eventing
  - sources.knative.dev/v1 (knative-eventing v1.9.0)
  - eventing.knative.dev/v1 (knative-eventing v1.9.0)

Kubernetes

$ kubectl version -o yaml
clientVersion:
  buildDate: "2023-10-30T00:22:50Z"
  compiler: gc
  gitCommit: 07a61d861519c45ef5c89bc22dda289328f29343
  gitTreeState: archive
  gitVersion: v1.27.7
  goVersion: go1.20.10
  major: "1"
  minor: "27"
  platform: darwin/arm64
kustomizeVersion: v5.0.1
serverVersion:
  buildDate: "2023-05-17T14:13:28Z"
  compiler: gc
  gitCommit: 7f6f68fdabc4df88cfea2dcf9a19b2b830f1e647
  gitTreeState: clean
  gitVersion: v1.27.2
  goVersion: go1.20.4
  major: "1"
  minor: "27"
  platform: linux/arm64

[oscerd]

Thanks for the detailed report. Cc @christophd @squakez @gansheer @claudio4j let's have a look next week if feasible

[squakez]

Thanks for reporting. I think it's a bug in the IsResolvable func, where we are expecting the variable to start and end with {{ and }} whereas here it's not the case as there are added parameters. I'm opening a new issue for that.