Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ClassNotFoundException for io.apicurio.registry.serde.avro.AvroKafkaDeserializer class in native version #18148

Closed
rmarting opened this issue Jun 25, 2021 · 7 comments
Closed

ClassNotFoundException for io.apicurio.registry.serde.avro.AvroKafkaDeserializer class in native version #18148

rmarting opened this issue Jun 25, 2021 · 7 comments
Labels
area/kafka area/mandrel kind/bug Something isn't working

Comments

@rmarting
Copy link

Describe the bug

An application using Apicurio Avro Serde classes to pub/sub messages from a Kafka cluster fails when it is compiled in native mode with a ClassNotFoundException of the class io.apicurio.registry.serde.avro.AvroKafkaDeserializer.

The same application compiled in jvm mode works successfully.

The stacktrace of the error is:

2021-06-25 08:43:57,512 ERROR [io.sma.rea.mes.provider] (main) SRMSG00230: Unable to create the publisher or subscriber during initialization: java.lang.IllegalArgumentException: SRMSG18010: Unable to create an instance of `io.apicurio.registry.serde.avro.AvroKafkaDeserializer`
	at io.smallrye.reactive.messaging.kafka.fault.DeserializerWrapper.createDelegateDeserializer(DeserializerWrapper.java:93)
	at io.smallrye.reactive.messaging.kafka.fault.DeserializerWrapper.<init>(DeserializerWrapper.java:65)
	at io.smallrye.reactive.messaging.kafka.impl.KafkaSource.<init>(KafkaSource.java:134)
	at io.smallrye.reactive.messaging.kafka.KafkaConnector.getPublisherBuilder(KafkaConnector.java:161)
	at io.smallrye.reactive.messaging.kafka.KafkaConnector_ClientProxy.getPublisherBuilder(KafkaConnector_ClientProxy.zig:277)
	at io.smallrye.reactive.messaging.impl.ConfiguredChannelFactory.createPublisherBuilder(ConfiguredChannelFactory.java:190)
	at io.smallrye.reactive.messaging.impl.ConfiguredChannelFactory.register(ConfiguredChannelFactory.java:153)
	at io.smallrye.reactive.messaging.impl.ConfiguredChannelFactory.initialize(ConfiguredChannelFactory.java:125)
	at io.smallrye.reactive.messaging.impl.ConfiguredChannelFactory_ClientProxy.initialize(ConfiguredChannelFactory_ClientProxy.zig:189)
	at java.util.Iterator.forEachRemaining(Iterator.java:133)
	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
	at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658)
	at io.smallrye.reactive.messaging.extension.MediatorManager.start(MediatorManager.java:181)
	at io.smallrye.reactive.messaging.extension.MediatorManager_ClientProxy.start(MediatorManager_ClientProxy.zig:282)
	at io.quarkus.smallrye.reactivemessaging.runtime.SmallRyeReactiveMessagingLifecycle.onApplicationStart(SmallRyeReactiveMessagingLifecycle.java:40)
	at io.quarkus.smallrye.reactivemessaging.runtime.SmallRyeReactiveMessagingLifecycle_Observer_onApplicationStart_4e8937813d9e8faff65c3c07f88fa96615b70e70.notify(SmallRyeReactiveMessagingLifecycle_Observer_onApplicationStart_4e8937813d9e8faff65c3c07f88fa96615b70e70.zig:111)
	at io.quarkus.arc.impl.EventImpl$Notifier.notifyObservers(EventImpl.java:283)
	at io.quarkus.arc.impl.EventImpl$Notifier.notify(EventImpl.java:268)
	at io.quarkus.arc.impl.EventImpl.fire(EventImpl.java:70)
	at io.quarkus.arc.runtime.ArcRecorder.fireLifecycleEvent(ArcRecorder.java:128)
	at io.quarkus.arc.runtime.ArcRecorder.handleLifecycleEvents(ArcRecorder.java:97)
	at io.quarkus.deployment.steps.LifecycleEventsBuildStep$startupEvent1144526294.deploy_0(LifecycleEventsBuildStep$startupEvent1144526294.zig:87)
	at io.quarkus.deployment.steps.LifecycleEventsBuildStep$startupEvent1144526294.deploy(LifecycleEventsBuildStep$startupEvent1144526294.zig:40)
	at io.quarkus.runner.ApplicationImpl.doStart(ApplicationImpl.zig:664)
	at io.quarkus.runtime.Application.start(Application.java:90)
	at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:100)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:66)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:42)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:119)
	at io.quarkus.runner.GeneratedMain.main(GeneratedMain.zig:29)
Caused by: java.lang.ClassNotFoundException: io.apicurio.registry.serde.avro.AvroKafkaDeserializer
	at com.oracle.svm.core.hub.ClassForNameSupport.forName(ClassForNameSupport.java:64)
	at java.lang.Class.forName(DynamicHub.java:1308)
	at org.apache.kafka.common.utils.Utils.loadClass(Utils.java:378)
	at org.apache.kafka.common.utils.Utils.newInstance(Utils.java:367)
	at io.smallrye.reactive.messaging.kafka.fault.DeserializerWrapper.createDelegateDeserializer(DeserializerWrapper.java:91)
	... 29 more

Application is developed with the following artifacts and versions:

  • Apicurio Avro Serde 2.0.1.Final (io.apicurio::apicurio-registry-serdes-avro-serde)
  • Avro 1.10.2

Expected behavior

To run exactly as the jvm version of the application.

Actual behavior

Failing with ClassNotFoundException of some classes from Apicurio Avro Serde dependency.

To Reproduce

The application is avaible in the upgrade-to-kafka-2.7-apicurio-2.0 branch of my git repo:

https://github.com/rmarting/kafka-clients-quarkus-sample/tree/upgrade-to-kafka-2.7-apicurio-2.0

Steps to reproduce the behavior:

  1. Compile in native mode:
❯ ./mvnw clean package -Pnative
[INFO] Scanning for projects...
[INFO] 
[INFO] ---------< io.jromanmartin.kafka:kafka-clients-quarkus-sample >---------
[INFO] Building kafka-clients-quarkus-sample 2.0.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
# omitted
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ kafka-clients-quarkus-sample ---
[INFO] Building jar: /home/rmarting/Workspaces/ws-streams/kafka-clients-quarkus-sample/target/kafka-clients-quarkus-sample-2.0.0-SNAPSHOT.jar
[INFO] 
[INFO] --- quarkus-maven-plugin:1.13.7.Final:build (default) @ kafka-clients-quarkus-sample ---
[INFO] [org.jboss.threads] JBoss Threads version 3.2.0.Final
[INFO] [io.quarkus.deployment.pkg.steps.JarResultBuildStep] Building native image source jar: /home/rmarting/Workspaces/ws-streams/kafka-clients-quarkus-sample/target/kafka-clients-quarkus-sample-2.0.0-SNAPSHOT-native-image-source-jar/kafka-clients-quarkus-sample-2.0.0-SNAPSHOT-runner.jar
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Building native image from /home/rmarting/Workspaces/ws-streams/kafka-clients-quarkus-sample/target/kafka-clients-quarkus-sample-2.0.0-SNAPSHOT-native-image-source-jar/kafka-clients-quarkus-sample-2.0.0-SNAPSHOT-runner.jar
[ERROR] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Unable to get GraalVM version from the native-image binary.
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildRunner] /home/rmarting/Software/mandrel/bin/native-image -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Dsun.nio.ch.maxUpdateArraySize=100 -J-Davro.disable.unsafe=true -J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -J-Dvertx.disableDnsResolver=true -J-Dio.netty.leakDetection.level=DISABLED -J-Dio.netty.allocator.maxOrder=1 -J-Duser.language=en -J-Duser.country=US -J-Dfile.encoding=UTF-8 --initialize-at-build-time= -H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy\$BySpaceAndTime -H:+JNI -H:+AllowFoldMethods -jar kafka-clients-quarkus-sample-2.0.0-SNAPSHOT-runner.jar -H:FallbackThreshold=0 -H:+ReportExceptionStackTraces -H:-AddAllCharsets -H:EnableURLProtocols=http -H:NativeLinkerOption=-no-pie --no-server -H:-UseServiceLoaderFeature -H:+StackTrace kafka-clients-quarkus-sample-2.0.0-SNAPSHOT-runner
Warning: Ignoring server-mode native-image argument --no-server.
[kafka-clients-quarkus-sample-2.0.0-SNAPSHOT-runner:521407]    classlist:   2,828.67 ms,  1.19 GB
[kafka-clients-quarkus-sample-2.0.0-SNAPSHOT-runner:521407]        (cap):     421.00 ms,  1.19 GB
[kafka-clients-quarkus-sample-2.0.0-SNAPSHOT-runner:521407]        setup:   1,908.88 ms,  1.19 GB
09:02:26,818 INFO  [org.jbo.threads] JBoss Threads version 3.2.0.Final
[kafka-clients-quarkus-sample-2.0.0-SNAPSHOT-runner:521407]     (clinit):     879.85 ms,  5.61 GB
[kafka-clients-quarkus-sample-2.0.0-SNAPSHOT-runner:521407]   (typeflow):  18,668.41 ms,  5.61 GB
[kafka-clients-quarkus-sample-2.0.0-SNAPSHOT-runner:521407]    (objects):  29,460.83 ms,  5.61 GB
[kafka-clients-quarkus-sample-2.0.0-SNAPSHOT-runner:521407]   (features):   1,197.54 ms,  5.61 GB
[kafka-clients-quarkus-sample-2.0.0-SNAPSHOT-runner:521407]     analysis:  51,961.22 ms,  5.61 GB
[kafka-clients-quarkus-sample-2.0.0-SNAPSHOT-runner:521407]     universe:   2,017.33 ms,  5.61 GB
[kafka-clients-quarkus-sample-2.0.0-SNAPSHOT-runner:521407]      (parse):   8,750.03 ms,  6.44 GB
[kafka-clients-quarkus-sample-2.0.0-SNAPSHOT-runner:521407]     (inline):   6,487.86 ms,  7.52 GB
[kafka-clients-quarkus-sample-2.0.0-SNAPSHOT-runner:521407]    (compile):  26,301.48 ms,  7.58 GB
[kafka-clients-quarkus-sample-2.0.0-SNAPSHOT-runner:521407]      compile:  44,250.67 ms,  7.58 GB
[kafka-clients-quarkus-sample-2.0.0-SNAPSHOT-runner:521407]        image:   9,084.08 ms,  7.55 GB
[kafka-clients-quarkus-sample-2.0.0-SNAPSHOT-runner:521407]        write:     692.47 ms,  7.55 GB
# Printing build artifacts to: kafka-clients-quarkus-sample-2.0.0-SNAPSHOT-runner.build_artifacts.txt
[kafka-clients-quarkus-sample-2.0.0-SNAPSHOT-runner:521407]      [total]: 113,053.80 ms,  7.55 GB
[INFO] [io.quarkus.deployment.QuarkusAugmentor] Quarkus augmentation completed in 116895ms
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  02:00 min
[INFO] Finished at: 2021-06-25T09:03:57+02:00
[INFO] ------------------------------------------------------------------------
  1. Execute application and found the exception
❯ ./target/kafka-clients-quarkus-sample-2.0.0-SNAPSHOT-runner 
__  ____  __  _____   ___  __ ____  ______ 
 --/ __ \/ / / / _ | / _ \/ //_/ / / / __/ 
 -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \   
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/   
2021-06-25 09:05:05,910 INFO  [io.sma.rea.mes.kafka] (main) SRMSG18229: Configured topics for channel 'messages': [messages]
2021-06-25 09:05:05,911 INFO  [io.sma.rea.mes.kafka] (main) SRMSG18234: Auto-commit disabled for channel messages
2021-06-25 09:05:05,911 ERROR [io.sma.rea.mes.provider] (main) SRMSG00230: Unable to create the publisher or subscriber during initialization: java.lang.IllegalArgumentException: SRMSG18010: Unable to create an instance of `io.apicurio.registry.serde.avro.AvroKafkaDeserializer`
	at io.smallrye.reactive.messaging.kafka.fault.DeserializerWrapper.createDelegateDeserializer(DeserializerWrapper.java:93)
	at io.smallrye.reactive.messaging.kafka.fault.DeserializerWrapper.<init>(DeserializerWrapper.java:65)
	at io.smallrye.reactive.messaging.kafka.impl.KafkaSource.<init>(KafkaSource.java:134)
	at io.smallrye.reactive.messaging.kafka.KafkaConnector.getPublisherBuilder(KafkaConnector.java:161)
	at io.smallrye.reactive.messaging.kafka.KafkaConnector_ClientProxy.getPublisherBuilder(KafkaConnector_ClientProxy.zig:277)
	at io.smallrye.reactive.messaging.impl.ConfiguredChannelFactory.createPublisherBuilder(ConfiguredChannelFactory.java:190)
	at io.smallrye.reactive.messaging.impl.ConfiguredChannelFactory.register(ConfiguredChannelFactory.java:153)
	at io.smallrye.reactive.messaging.impl.ConfiguredChannelFactory.initialize(ConfiguredChannelFactory.java:125)
	at io.smallrye.reactive.messaging.impl.ConfiguredChannelFactory_ClientProxy.initialize(ConfiguredChannelFactory_ClientProxy.zig:189)
	at java.util.Iterator.forEachRemaining(Iterator.java:133)
	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
	at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658)
	at io.smallrye.reactive.messaging.extension.MediatorManager.start(MediatorManager.java:181)
	at io.smallrye.reactive.messaging.extension.MediatorManager_ClientProxy.start(MediatorManager_ClientProxy.zig:282)
	at io.quarkus.smallrye.reactivemessaging.runtime.SmallRyeReactiveMessagingLifecycle.onApplicationStart(SmallRyeReactiveMessagingLifecycle.java:40)
	at io.quarkus.smallrye.reactivemessaging.runtime.SmallRyeReactiveMessagingLifecycle_Observer_onApplicationStart_4e8937813d9e8faff65c3c07f88fa96615b70e70.notify(SmallRyeReactiveMessagingLifecycle_Observer_onApplicationStart_4e8937813d9e8faff65c3c07f88fa96615b70e70.zig:111)
	at io.quarkus.arc.impl.EventImpl$Notifier.notifyObservers(EventImpl.java:283)
	at io.quarkus.arc.impl.EventImpl$Notifier.notify(EventImpl.java:268)
	at io.quarkus.arc.impl.EventImpl.fire(EventImpl.java:70)
	at io.quarkus.arc.runtime.ArcRecorder.fireLifecycleEvent(ArcRecorder.java:128)
	at io.quarkus.arc.runtime.ArcRecorder.handleLifecycleEvents(ArcRecorder.java:97)
	at io.quarkus.deployment.steps.LifecycleEventsBuildStep$startupEvent1144526294.deploy_0(LifecycleEventsBuildStep$startupEvent1144526294.zig:87)
	at io.quarkus.deployment.steps.LifecycleEventsBuildStep$startupEvent1144526294.deploy(LifecycleEventsBuildStep$startupEvent1144526294.zig:40)
	at io.quarkus.runner.ApplicationImpl.doStart(ApplicationImpl.zig:664)
	at io.quarkus.runtime.Application.start(Application.java:90)
	at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:100)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:66)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:42)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:119)
	at io.quarkus.runner.GeneratedMain.main(GeneratedMain.zig:29)
Caused by: java.lang.ClassNotFoundException: io.apicurio.registry.serde.avro.AvroKafkaDeserializer
	at com.oracle.svm.core.hub.ClassForNameSupport.forName(ClassForNameSupport.java:64)
	at java.lang.Class.forName(DynamicHub.java:1308)
	at org.apache.kafka.common.utils.Utils.loadClass(Utils.java:378)
	at org.apache.kafka.common.utils.Utils.newInstance(Utils.java:367)
	at io.smallrye.reactive.messaging.kafka.fault.DeserializerWrapper.createDelegateDeserializer(DeserializerWrapper.java:91)
	... 29 more
  1. Compile in jvm mode:
❯ ./mvnw clean package
[INFO] Scanning for projects...
[INFO] 
[INFO] ---------< io.jromanmartin.kafka:kafka-clients-quarkus-sample >---------
[INFO] Building kafka-clients-quarkus-sample 2.0.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
# omitted
[INFO] --- quarkus-maven-plugin:1.13.7.Final:build (default) @ kafka-clients-quarkus-sample ---
[INFO] [org.jboss.threads] JBoss Threads version 3.2.0.Final
[INFO] [io.quarkus.deployment.pkg.steps.JarResultBuildStep] Building fat jar: /home/rmarting/Workspaces/ws-streams/kafka-clients-quarkus-sample/target/kafka-clients-quarkus-sample-2.0.0-SNAPSHOT-runner.jar
[WARNING] [io.quarkus.deployment.pkg.steps.JarResultBuildStep] Dependencies with duplicate files detected. The dependencies [com.sun.activation:jakarta.activation::jar:1.2.1(compile), jakarta.activation:jakarta.activation-api::jar:1.2.1(compile)] contain duplicate files, e.g. javax/activation/ActivationDataFlavor.class
[WARNING] [io.quarkus.deployment.pkg.steps.JarResultBuildStep] Dependencies with duplicate files detected. The dependencies [com.fasterxml.jackson.jaxrs:jackson-jaxrs-json-provider::jar:2.12.3(compile), com.fasterxml.jackson.jaxrs:jackson-jaxrs-base::jar:2.12.3(compile), com.fasterxml.jackson.module:jackson-module-jaxb-annotations::jar:2.12.3(compile)] contain duplicate files, e.g. META-INF/versions/11/module-info.class
[WARNING] [io.quarkus.deployment.pkg.steps.JarResultBuildStep] Dependencies with duplicate files detected. The dependencies [org.bouncycastle:bcpkix-jdk15on::jar:1.68(compile), org.bouncycastle:bcprov-jdk15on::jar:1.68(compile)] contain duplicate files, e.g. META-INF/versions/9/module-info.class
[INFO] [io.quarkus.deployment.QuarkusAugmentor] Quarkus augmentation completed in 5031ms
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  8.880 s
[INFO] Finished at: 2021-06-25T09:05:53+02:00
[INFO] ------------------------------------------------------------------------
  1. Execute application and not found the exception:
❯ java -jar ./target/kafka-clients-quarkus-sample-2.0.0-SNAPSHOT-runner.jar 
__  ____  __  _____   ___  __ ____  ______ 
 --/ __ \/ / / / _ | / _ \/ //_/ / / / __/ 
 -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \   
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/   
2021-06-25 09:06:58,765 INFO  [io.sma.rea.mes.kafka] (main) SRMSG18229: Configured topics for channel 'messages': [messages]
2021-06-25 09:06:58,777 INFO  [io.sma.rea.mes.kafka] (main) SRMSG18234: Auto-commit disabled for channel messages
2021-06-25 09:06:58,902 INFO  [org.apa.kaf.cli.con.ConsumerConfig] (main) ConsumerConfig values: 
	allow.auto.create.topics = true
	auto.commit.interval.ms = 5000
	auto.offset.reset = latest
	bootstrap.servers = [my-kafka-kafka-bootstrap:9092]
	check.crcs = true
	client.dns.lookup = use_all_dns_ips
	client.id = kafka-consumer-messages
	client.rack = 
	connections.max.idle.ms = 540000
	default.api.timeout.ms = 60000
	enable.auto.commit = false
	exclude.internal.topics = true
	fetch.max.bytes = 52428800
	fetch.max.wait.ms = 500
	fetch.min.bytes = 1
	group.id = kafka-client-quarkus-consumer-mp-incoming-channel
	group.instance.id = null
	heartbeat.interval.ms = 3000
	interceptor.classes = []
	internal.leave.group.on.close = true
	internal.throw.on.fetch.stable.offset.unsupported = false
	isolation.level = read_uncommitted
	key.deserializer = class io.smallrye.reactive.messaging.kafka.fault.DeserializerWrapper
	max.partition.fetch.bytes = 1048576
	max.poll.interval.ms = 300000
	max.poll.records = 500
	metadata.max.age.ms = 300000
	metric.reporters = []
	metrics.num.samples = 2
	metrics.recording.level = INFO
	metrics.sample.window.ms = 30000
	partition.assignment.strategy = [class org.apache.kafka.clients.consumer.RangeAssignor]
	receive.buffer.bytes = 65536
	reconnect.backoff.max.ms = 10000
	reconnect.backoff.ms = 50
	request.timeout.ms = 30000
	retry.backoff.ms = 100
	sasl.client.callback.handler.class = null
	sasl.jaas.config = [hidden]
	sasl.kerberos.kinit.cmd = /usr/bin/kinit
	sasl.kerberos.min.time.before.relogin = 60000
	sasl.kerberos.service.name = null
	sasl.kerberos.ticket.renew.jitter = 0.05
	sasl.kerberos.ticket.renew.window.factor = 0.8
	sasl.login.callback.handler.class = null
	sasl.login.class = null
	sasl.login.refresh.buffer.seconds = 300
	sasl.login.refresh.min.period.seconds = 60
	sasl.login.refresh.window.factor = 0.8
	sasl.login.refresh.window.jitter = 0.05
	sasl.mechanism = SCRAM-SHA-512
	security.protocol = SASL_PLAINTEXT
	security.providers = null
	send.buffer.bytes = 131072
	session.timeout.ms = 10000
	socket.connection.setup.timeout.max.ms = 127000
	socket.connection.setup.timeout.ms = 10000
	ssl.cipher.suites = null
	ssl.enabled.protocols = [TLSv1.2, TLSv1.3]
	ssl.endpoint.identification.algorithm = https
	ssl.engine.factory.class = null
	ssl.key.password = null
	ssl.keymanager.algorithm = SunX509
	ssl.keystore.certificate.chain = null
	ssl.keystore.key = null
	ssl.keystore.location = null
	ssl.keystore.password = null
	ssl.keystore.type = JKS
	ssl.protocol = TLSv1.3
	ssl.provider = null
	ssl.secure.random.implementation = null
	ssl.trustmanager.algorithm = PKIX
	ssl.truststore.certificates = null
	ssl.truststore.location = null
	ssl.truststore.password = null
	ssl.truststore.type = JKS
	value.deserializer = class io.smallrye.reactive.messaging.kafka.fault.DeserializerWrapper

Configuration

Smallrye properties related with Apicurio Avro Serde classes:

# Configure the Kafka source (we read from it)
mp.messaging.incoming.messages.connector=smallrye-kafka
mp.messaging.incoming.messages.group.id=${app.consumer.groupId}-mp-incoming-channel
mp.messaging.incoming.messages.topic=messages
mp.messaging.incoming.messages.key.deserializer=org.apache.kafka.common.serialization.StringDeserializer
mp.messaging.incoming.messages.value.deserializer=io.apicurio.registry.serde.avro.AvroKafkaDeserializer
mp.messaging.incoming.messages.properties.partition.assignment.strategy=org.apache.kafka.clients.consumer.RoundRobinAssignor
mp.messaging.incoming.messages.apicurio.registry.url=${apicurio.registry.url}
#mp.messaging.incoming.messages.apicurio.registry.avro-datum-provider=io.apicurio.registry.utils.serde.avro.ReflectAvroDatumProvider
mp.messaging.incoming.messages.apicurio.registry.use-specific-avro-reader=true

Environment:

Output of uname -a or ver

❯ uname -a
Linux redhat 5.12.10-200.fc33.x86_64 #1 SMP Thu Jun 10 14:19:48 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Output of java -version

❯ java -version
openjdk version "11.0.11" 2021-04-20
OpenJDK Runtime Environment GraalVM CE 21.1.0 (build 11.0.11+8-jvmci-21.1-b05)
OpenJDK 64-Bit Server VM GraalVM CE 21.1.0 (build 11.0.11+8-jvmci-21.1-b05, mixed mode, sharing)

GraalVM version (if different from Java)

❯ ./bin/native-image --version
native-image 21.1.0.0-Final (Mandrel Distribution) (Java Version 11.0.11+9)

Quarkus version or git rev

Quarkus 1.13.7.Final

Build tool (ie. output of mvnw --version or gradlew --version)

❯ ./mvnw --version
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: /home/rmarting/.m2/wrapper/dists/apache-maven-3.6.3-bin/1iopthnavndlasol9gbrbg6bf2/apache-maven-3.6.3
Java version: 11.0.11, vendor: Red Hat, Inc., runtime: /usr/lib/jvm/java-11-openjdk-11.0.11.0.9-4.fc33.x86_64
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "5.12.10-200.fc33.x86_64", arch: "amd64", family: "unix"

Additional context

Avro and Apicurio dependencies:

        <!-- Avro Schemas Dependency -->
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-avro</artifactId>
        </dependency>

        <!-- Apicurio Serializers/Deserializers -->
        <dependency>
            <groupId>io.apicurio</groupId>
            <artifactId>apicurio-registry-serdes-avro-serde</artifactId>
            <version>2.0.1.Final</version>
        </dependency>
@rmarting rmarting added the kind/bug Something isn't working label Jun 25, 2021
@quarkus-bot
Copy link

quarkus-bot bot commented Jun 25, 2021

/cc @cescoffier, @galderz, @zakkak

@rmarting rmarting mentioned this issue Jun 25, 2021
Closed
@zakkak
Copy link
Contributor

zakkak commented Jun 25, 2021 via email

@rmarting
Copy link
Author

The issue is also reproducible with both Mandrel versions compatible with Quarkus 1.13:

Mandrel 20.3.2.0-Final:

❯ ./bin/native-image --version
GraalVM Version 20.3.2.0-Final (Mandrel Distribution) (Java Version 11.0.11+9)

Maven packaging:

[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Building native image from /home/rmarting/Workspaces/ws-streams/kafka-clients-quarkus-sample/target/kafka-clients-quarkus-sample-2.0.0-SNAPSHOT-native-image-source-jar/kafka-clients-quarkus-sample-2.0.0-SNAPSHOT-runner.jar
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Running Quarkus native-image plugin on GraalVM Version 20.3.2.0-Final (Mandrel Distribution) (Java Version 11.0.11+9)

Mandrel 21.0.0.0-Final:

❯ ./bin/native-image --version
GraalVM Version 21.0.0.0-Final (Mandrel Distribution) (Java Version 11.0.10+9)

Maven packaging:

[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Building native image from /home/rmarting/Workspaces/ws-streams/kafka-clients-quarkus-sample/target/kafka-clients-quarkus-sample-2.0.0-SNAPSHOT-native-image-source-jar/kafka-clients-quarkus-sample-2.0.0-SNAPSHOT-runner.jar
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Running Quarkus native-image plugin on GraalVM Version 21.0.0.0-Final (Mandrel Distribution) (Java Version 11.0.10+9)

So it seems that it is not related directly with the Mandrel version itself.

@geoand
Copy link
Contributor

geoand commented Jun 25, 2021

cc @Ladicek

@Ladicek
Copy link
Contributor

Ladicek commented Jun 28, 2021

Support for Apicurio Registry 2.x Avro libraries was first added in #16473, which was first present in Quarkus 2.0.0.Alpha2. (Quarkus 2.0.0.Final also has #17725, so it's even better.)

Unfortunately, Quarkus 1.13 doesn't support Apicurio Registry 2.x libraries at all.

@rmarting
Copy link
Author

Thanks @Ladicek for your update.

I tested it with Quarkus 2.0.0.Final and with the following Mandrel distributions:

❯ ./mandrel/bin/native-image --version
GraalVM Version 20.3.2.0-Final (Mandrel Distribution) (Java Version 11.0.11+9)
❯ ./mandrel/bin/native-image --version
native-image 21.1.0.0-Final (Mandrel Distribution) (Java Version 11.0.11+9)

In both cases the application in native mode works successfully. To complete the migration I have to add the io.quarkus:quarkus-apicurio-registry-avro extension in the pom.xml file.

I have to test using the io.quarkus:quarkus-bom bom file instead of io.quarkus:quarkus-universe-bom because at this moment is not available in the Maven repos.

IMHO this bug could be closed because it is fixed for Quarkus 2 and Apicurio Registry 2.

@Ladicek
Copy link
Contributor

Ladicek commented Jun 29, 2021

Thanks for confirmation! FYI, quarkus-universe-bom for 2.0 should be released shortly. I don't know exactly when, but real soon :-)

@Ladicek Ladicek closed this as completed Jun 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/kafka area/mandrel kind/bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@Ladicek @zakkak @geoand @rmarting