Kafka Avro support does not work in native mode

[cescoffier]

Describe the bug
Avro support does not work in native.
Avro serialization uses Unsafe to write fields.

To Reproduce
Steps to reproduce the behavior:

1. Use code from PR Increase kafka client test coverage #8237
2. Register the serializers in the Kafka processor
3. Create the IT class for the Avro tester case in the integration-test
4. Run the tests, boom.

[gwenneg]

Maybe io.quarkus.deployment.builditem.nativeimage.UnsafeAccessedFieldBuildItem could help here.

[schulzp]

@cescoffier, we ran into the same issue and this is what I found out: org.apache.avro:avro utilizes sun.misc.Unsafe to get memory offsets for properties of objects inspected via reflection. GraalVM is only capable of handling static usages of sun.misc.Unsafe, however.

In order to work around this, I wrote a custom (de-)serializer pair that skips reflection and uses explicit code-generated classes. You might be able to go from there and write a (de-)serializer pair which uses GenericRecord instead.

[rmarting]

I am working on a project with Quarkus 1.13.7.Final, Smallrye Reactive, Avro 1.10.2, and Apicurio Serdes 2.0.1.Final.

This project is working successfully when it is deployed using jvm, however when I build and deploy the native version, I got the following exception:

2021-06-24 15:18:24,772 INFO  [io.sma.rea.mes.kafka] (main) SRMSG18237: Setting client.id for Kafka producer to kafka-producer-eda-alerts
2021-06-24 15:18:24,773 ERROR [io.sma.rea.mes.provider] (main) SRMSG00230: Unable to create the publisher or subscriber during initialization: org.apache.kafka.common.config.ConfigException: Invalid value io.apicurio.registry.serde.avro.AvroKafkaSerializer for configuration value.serializer: Class io.apicurio.registry.serde.avro.AvroKafkaSerializer could not be found.
	at org.apache.kafka.common.config.ConfigDef.parseType(ConfigDef.java:729)
	at org.apache.kafka.common.config.ConfigDef.parseValue(ConfigDef.java:475)
	at org.apache.kafka.common.config.ConfigDef.parse(ConfigDef.java:468)
	at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:108)
	at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:129)
	at org.apache.kafka.clients.producer.ProducerConfig.<init>(ProducerConfig.java:536)
	at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:330)
	at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:274)
	at io.vertx.kafka.client.producer.impl.KafkaWriteStreamImpl.create(KafkaWriteStreamImpl.java:51)
	at io.vertx.kafka.client.producer.KafkaWriteStream.create(KafkaWriteStream.java:92)
	at io.smallrye.reactive.messaging.kafka.impl.KafkaSink.<init>(KafkaSink.java:68)
	at io.smallrye.reactive.messaging.kafka.KafkaConnector.getSubscriberBuilder(KafkaConnector.java:198)
	at io.smallrye.reactive.messaging.kafka.KafkaConnector_ClientProxy.getSubscriberBuilder(KafkaConnector_ClientProxy.zig:128)
	at io.smallrye.reactive.messaging.impl.ConfiguredChannelFactory.createSubscriberBuilder(ConfiguredChannelFactory.java:207)
	at io.smallrye.reactive.messaging.impl.ConfiguredChannelFactory.register(ConfiguredChannelFactory.java:164)
	at io.smallrye.reactive.messaging.impl.ConfiguredChannelFactory.initialize(ConfiguredChannelFactory.java:125)
	at io.smallrye.reactive.messaging.impl.ConfiguredChannelFactory_ClientProxy.initialize(ConfiguredChannelFactory_ClientProxy.zig:189)
	at java.util.Iterator.forEachRemaining(Iterator.java:133)
	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
	at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658)
	at io.smallrye.reactive.messaging.extension.MediatorManager.start(MediatorManager.java:181)
	at io.smallrye.reactive.messaging.extension.MediatorManager_ClientProxy.start(MediatorManager_ClientProxy.zig:282)
	at io.quarkus.smallrye.reactivemessaging.runtime.SmallRyeReactiveMessagingLifecycle.onApplicationStart(SmallRyeReactiveMessagingLifecycle.java:40)
	at io.quarkus.smallrye.reactivemessaging.runtime.SmallRyeReactiveMessagingLifecycle_Observer_onApplicationStart_4e8937813d9e8faff65c3c07f88fa96615b70e70.notify(SmallRyeReactiveMessagingLifecycle_Observer_onApplicationStart_4e8937813d9e8faff65c3c07f88fa96615b70e70.zig:111)
	at io.quarkus.arc.impl.EventImpl$Notifier.notifyObservers(EventImpl.java:283)
	at io.quarkus.arc.impl.EventImpl$Notifier.notify(EventImpl.java:268)
	at io.quarkus.arc.impl.EventImpl.fire(EventImpl.java:70)
	at io.quarkus.arc.runtime.ArcRecorder.fireLifecycleEvent(ArcRecorder.java:128)
	at io.quarkus.arc.runtime.ArcRecorder.handleLifecycleEvents(ArcRecorder.java:97)
	at io.quarkus.deployment.steps.LifecycleEventsBuildStep$startupEvent1144526294.deploy_0(LifecycleEventsBuildStep$startupEvent1144526294.zig:87)
	at io.quarkus.deployment.steps.LifecycleEventsBuildStep$startupEvent1144526294.deploy(LifecycleEventsBuildStep$startupEvent1144526294.zig:40)
	at io.quarkus.runner.ApplicationImpl.doStart(ApplicationImpl.zig:653)
	at io.quarkus.runtime.Application.start(Application.java:90)
	at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:100)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:66)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:42)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:119)
	at io.quarkus.runner.GeneratedMain.main(GeneratedMain.zig:29)

Is this error related with this bug? I mean, as Avro is not supported in native, then we could find this kind of issue.

My project is available here

[cescoffier]

Actually, this issue has been fixed a long time ago (like a year ago).

@rmanibus, which version of Apicurio and Kafka are you using? Can you open a separate issue?
It seems it can't find the apicurio class.

[rmarting]

@cescoffier it seems that it is something related with Apicurio Avro Serde classes instead of Avro itself. I opened a new issue #18148 to track it.