Zilla OpenAPI not supporting filesystem catalog

[casanova-thiago]

Describe the bug
I was trying to use my OpenAPI schema with Zilla 0.9.91. I'm very new to Zilla and many of the k8s/helm concepts so the example I interacted with the most was the http.kafka.async, so I used this folder as my application dir. So forgive me for the huge mix my setup was.

To Reproduce
Steps to reproduce the behaviour:

1. clone zilla-examples repository
2. cd into http.kafka.async folder
3. update all original files with the ones attached here
4. run ./setup.sh
5. check zilla pod logs

Expected behavior
Zilla pod starts and loads all configurations without any issues.

Screenshots
N/A

Zilla Environment:

setup.sh

#!/bin/bash
set -ex

# Install Zilla to the Kubernetes cluster with helm and wait for the pod to start up
ZILLA_CHART="${ZILLA_CHART:-oci://ghcr.io/aklivity/charts/zilla}"
ZILLA_VERSION="${ZILLA_VERSION:-0.9.91}"
NAMESPACE="${NAMESPACE:-zilla-http-kafka-async}"

kubectl get ns $NAMESPACE || kubectl create ns $NAMESPACE
kubectl create configmap protobuf-files --from-file=./spec/ -n $NAMESPACE -o yaml --dry-run=client | kubectl apply -f -

helm upgrade --install zilla $ZILLA_CHART --version $ZILLA_VERSION --namespace $NAMESPACE --create-namespace --wait \
    --values values.yaml \
    --set-file zilla\\.yaml=zilla.yaml \

# Install Kafka to the Kubernetes cluster with helm and wait for the pod to start up
helm upgrade --install kafka chart --namespace $NAMESPACE --create-namespace --wait

# Create the items-requests and items-responses topics in Kafka
KAFKA_POD=$(kubectl get pods --namespace $NAMESPACE --selector app.kubernetes.io/instance=kafka -o name)
kubectl exec --namespace $NAMESPACE "$KAFKA_POD" -- \
    /opt/bitnami/kafka/bin/kafka-topics.sh \
        --bootstrap-server localhost:9092 \
        --create \
        --topic items-requests \
        --if-not-exists
kubectl exec --namespace $NAMESPACE "$KAFKA_POD" -- \
    /opt/bitnami/kafka/bin/kafka-topics.sh \
        --bootstrap-server localhost:9092 \
        --create \
        --topic items-responses \
        --if-not-exists

# Start port forwarding
kubectl port-forward --namespace $NAMESPACE service/zilla 7114 7143 > /tmp/kubectl-zilla.log 2>&1 &
kubectl port-forward --namespace $NAMESPACE service/kafka 9092 29092 > /tmp/kubectl-kafka.log 2>&1 &
until nc -z localhost 7114; do sleep 1; done
until nc -z localhost 7143; do sleep 1; done
until nc -z localhost 9092; do sleep 1; done

Attach the values.yaml config file:

livenessProbePort: 7114
readinessProbePort: 7114

args: ["start", "-v", "-e", "-Pzilla.binding.kafka.client.produce.record.framing.size=1024"]

service:
  ports:
    - port: 7114
      name: http
    - port: 7143
      name: https

volumeMounts:
  - name: proto-files
    mountPath: /etc/zilla/spec

volumes:
  - name: proto-files
    configMap:
      name: protobuf-files

Attach the zilla.yaml config file:

---
name: example
catalogs:
  host_filesystem:
    type: filesystem
    options:
      subjects:
        api_schema:
          path: spec/api_schema.yaml
        kafka_schema:
          path: spec/kafka_schema.yaml
bindings:
  north_rest_server:
    type: openapi
    kind: server
    options:
      specs:
        api-rest-spec:
          catalog:
            host_filesystem:
              subject: api_schema
              version: latest
          servers:
            - url: localhost:8080
    exit: north_rest_kafka_proxy
    telemetry:
      metrics:
        - stream.*
        - http.*
  north_rest_kafka_proxy:
    type: openapi-asyncapi
    kind: proxy
    options:
      specs:
        openapi:
          api-rest-spec:
            catalog:
              host_filesystem:
                subject: api_schema
                version: latest
        asyncapi:
          api-kafka-spec:
            catalog:
              host_filesystem:
                subject: kafka_schema
                version: latest
    routes:
      - when:
          - api-id: api-rest-spec
        exit: south_kafka_client
        with:
          api-id: api-kafka-spec
    telemetry:
      metrics:
        - stream.*
  south_kafka_client:
    type: asyncapi
    kind: client
    options:
      specs:
        api-kafka-spec:
          catalog:
            host_filesystem:
              subject: kafka_schema
              version: latest
          servers:
            - host: kafka:9092
    telemetry:
      metrics:
        - stream.*
telemetry:
  exporters:
    stdout_logs_exporter:
      type: stdout

Attach the zilla dump pcap file:

ZILLA_INCUBATOR_ENABLED=true ./zilla dump --verbose --write /tmp/zilla.pcap

Pod is not running, cant execute.
zilla-describe.txt
zilla-error.txt

[vordimous]

NPE from the stack on startup found in the zilla-error.txt file:

java.lang.NullPointerException: Cannot invoke "io.aklivity.zilla.runtime.binding.openapi.asyncapi.internal.config.OpenapiAsyncapiBindingConfig.detach()" because "apiBinding" is null
    at [email protected]/io.aklivity.zilla.runtime.binding.openapi.asyncapi.internal.streams.OpenapiAsyncapiProxyFactory.detach(OpenapiAsyncapiProxyFactory.java:142)

[casanova-thiago]

noticed something interesting, I tried a couple of different versions 0.9.89 gave some errors related to my kafka-async API schema, I fixed them when I ran the setup.sh, and zilla started, with no errors, but a couple of seconds later it gets shutdown and restarts. Not sure if it is related, but adding it here.

[jfallows]

The NPE is a side effect on shutdown when startup has already failed.

In zilla-errors.txt, see line 100.

java.lang.IllegalAccessError: class io.aklivity.zilla.runtime.binding.openapi.asyncapi.internal.config.OpenapiAsyncapiBindingConfig (in module io.aklivity.zilla.runtime.binding.openapi.asyncapi) cannot access class io.aklivity.zilla.runtime.binding.asyncapi.internal.model.parser.AsyncapiParser (in module io.aklivity.zilla.runtime.binding.asyncapi) because module io.aklivity.zilla.runtime.binding.asyncapi does not export io.aklivity.zilla.runtime.binding.asyncapi.internal.model.parser to module io.aklivity.zilla.runtime.binding.openapi.asyncapi
    at [email protected]/io.aklivity.zilla.runtime.binding.openapi.asyncapi.internal.config.OpenapiAsyncapiBindingConfig.<init>(OpenapiAsyncapiBindingConfig.java:89)

Looks like the module-info was not updated when the AsyncapiParser was refactored to a different Java package.

If you would like to prevent this error on 0.9.92 then set the following JAVA_OPTIONS where zilla is running, so it will pick up the java command line parameters to temporarily provide access to AsyncapiParser package for openapi-asyncapi binding.

JAVA_OPTIONS="--add-opens io.aklivity.zilla.runtime.binding.asyncapi/io.aklivity.zilla.runtime.binding.asyncapi.internal.model.parser=io.aklivity.zilla.runtime.binding.openapi.asyncapi"

Separately, we'll make the corresponding fix so that this workaround can be removed.