Nested enums in avro schema may be produced but not consumed

[anchou-bockhorn]

Describe the bug

I have an avro schema, that defines a record, that contains another record with a field of type enum:

"type": "record",
  "name": "PersonUpdated",
  "namespace": "ch.abraxas.steuern.polaris.personenregister.api.event",
  "fields": [
    {
      "name": "partner",
      "default": null,
      "type": [
        "null",
        {
          "type": "record",
          "name": "NatuerlichePerson",
          "fields": [
            {
              "name": "geschlecht",
              "type": {
                "type": "enum",
                "name": "Geschlecht",
                "namespace": "ch.abraxas.steuern.polaris.personenregister.api.event",
                "symbols": [
                  "UNBESTIMMT"
                ]
...

Producing such a record works fine. Also autoregistering at schema registry works fine.

But consumptions of such a record fails with:
A failure has been reported for Kafka topics '[movies]': org.apache.avro.SchemaParseException: "Geschlecht" is not a defined name. The type of the "geschlecht" field must be a defined name or a {"type": ...} expression.

If the enum is defined on top level, as field of record ch.abraxas.steuern.polaris.personenregister.api.eventPersonUpdated, then all works fine.

Expected behavior

With Quarkus 2.9.1.Final all worked fine

Actual behavior

With Quarkus 2.13.3.Final only producing works. Consumption of a record produced by the same application fails with:
A failure has been reported for Kafka topics '[movies]': org.apache.avro.SchemaParseException: "Geschlecht" is not a defined name. The type of the "geschlecht" field must be a defined name or a {"type": ...} expression.

How to Reproduce?

Reproducer:
[email protected]:anchou-bockhorn/reproducer_avro_issue.git

Steps:

1. ./mvnw quarkus:dev # Start application
2. curl -X GET http://localhost:8080/movies # produce record
3. curl -N http://localhost:8080/consumed-movies # consume record

Output of uname -a or ver

Linux axabo-Lenovo 5.15.0-46-generic #49~20.04.1-Ubuntu SMP Thu Aug 4 19:15:44 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Output of java -version

openjdk version "11.0.16.1" 2022-08-12 LTS OpenJDK Runtime Environment Corretto-11.0.16.9.1 (build 11.0.16.1+9-LTS) OpenJDK 64-Bit Server VM Corretto-11.0.16.9.1 (build 11.0.16.1+9-LTS, mixed mode)

GraalVM version (if different from Java)

Quarkus version or git rev

2.13.3.Final

Build tool (ie. output of mvnw --version or gradlew --version)

Apache Maven 3.8.6 (84538c9988a25aec085021c365c560670ad80f63) Maven home: /home/axabo/.m2/wrapper/dists/apache-maven-3.8.6-bin/67568434/apache-maven-3.8.6 Java version: 11.0.16.1, vendor: Amazon.com Inc., runtime: /opt/corretto-11-jdk Default locale: en_US, platform encoding: UTF-8 OS name: "linux", version: "5.15.0-46-generic", arch: "amd64", family: "unix"

Additional information

No response

[quarkus-bot]

/cc @alesj, @cescoffier, @ozangunalp

[ozangunalp]

@anchou-bockhorn thanks for the reproducer, and sorry to keep you waiting. It was hard for me to pinpoint the reason. There seems to be a bug in the apicurio avro serde while registering the schema with references (IIUC Apicurio/apicurio-registry#2676 is the one affecting your case).

You can bump your quarkus version to 2.14.0.Final, which depends on the version 2.3.1.Final apicurio avro serde, with the fix.
You can also force a specific version of the apicurio avro serde dependency in your pom. But because of the complex dependency structure, it is a bit annoying to do :

<dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-apicurio-registry-avro</artifactId>
    <exclusions>
        <exclusion>
            <groupId>io.apicurio</groupId>
            <artifactId>apicurio-common-rest-client-vertx</artifactId>
        </exclusion>
        <exclusion>
            <groupId>io.apicurio</groupId>
            <artifactId>apicurio-registry-serdes-avro-serde</artifactId>
        </exclusion>
        <exclusion>
            <groupId>io.apicurio</groupId>
            <artifactId>apicurio-registry-client</artifactId>
        </exclusion>
    </exclusions>
</dependency>
<dependency>
    <groupId>io.apicurio</groupId>
    <artifactId>apicurio-registry-serdes-avro-serde</artifactId>
    <version>2.3.1.Final</version>
    <exclusions>
        <exclusion>
            <groupId>io.apicurio</groupId>
            <artifactId>apicurio-common-rest-client-jdk</artifactId>
        </exclusion>
    </exclusions>
</dependency>
<dependency>
    <groupId>io.apicurio</groupId>
    <artifactId>apicurio-registry-client</artifactId>
    <version>2.3.1.Final</version>
</dependency>
<dependency>
    <groupId>io.apicurio</groupId>
    <artifactId>apicurio-common-rest-client-vertx</artifactId>
    <version>0.1.13.Final</version>
</dependency>

[cescoffier]

@ozangunalp , Should we close this issue?

[anchou-bockhorn]

@ozangunalp many thx for the fix. I'll try it asap.

[cescoffier]

Closing - please reopen if the described fix does not work for you.