Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add apicurio protobuf support #37981

Closed
wants to merge 5 commits into from
Closed

Add apicurio protobuf support #37981

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
bafbe39
Add confluent json serde support
carlesarnal Dec 20, 2023
46165ff
Add protobuf extension for apicurio registry
carlesarnal Dec 20, 2023
710f241
Add apicurio protobuf integration tests
carlesarnal Dec 21, 2023
ede45bd
Add protobuf native integration tests for apicurio serdes support
carlesarnal Jan 31, 2024
267407b
Make test error reproducible
carlesarnal Feb 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 15 additions & 2 deletions bom/application/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1405,6 +1405,16 @@
<artifactId>quarkus-apicurio-registry-json-schema-deployment</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-apicurio-registry-protobuf</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-apicurio-registry-protobuf-deployment</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-confluent-registry-common</artifactId>
Expand Down Expand Up @@ -3420,6 +3430,11 @@
<artifactId>apicurio-registry-serdes-jsonschema-serde</artifactId>
<version>${apicurio-registry.version}</version>
</dependency>
<dependency>
<groupId>io.apicurio</groupId>
<artifactId>apicurio-registry-serdes-protobuf-serde</artifactId>
<version>${apicurio-registry.version}</version>
</dependency>
<dependency>
<groupId>io.apicurio</groupId>
<artifactId>apicurio-common-rest-client-vertx</artifactId>
Expand Down Expand Up @@ -6315,8 +6330,6 @@
<!-- we don't want our BOM to enforce Okhttp dependencies -->
<key>com.squareup.okhttp3:*</key>
<key>com.squareup.okhttp:*</key>
<key>com.squareup.okio:*</key>

<!-- jboss-parent:40 still manages jdk-misc, but does not define version.jdk-misc anymore. -->
<!-- We can just remove it because is not needed anyway. -->
<!-- The exclusion can be removed once all imported BOMs depend on jboss-parent:41+ -->
Expand Down
1 change: 1 addition & 0 deletions core/deployment/src/main/java/io/quarkus/deployment/Feature.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ public enum Feature {
AZURE_FUNCTIONS,
APICURIO_REGISTRY_AVRO,
APICURIO_REGISTRY_JSON_SCHEMA,
APICURIO_REGISTRY_PROTOBUF,
AWT,
CACHE,
CDI,
Expand Down
13 changes: 13 additions & 0 deletions devtools/bom-descriptor-json/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,19 @@
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-apicurio-registry-protobuf</artifactId>
<version>${project.version}</version>
<type>pom</type>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>*</groupId>
<artifactId>*</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-arc</artifactId>
Expand Down
13 changes: 13 additions & 0 deletions docs/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -229,6 +229,19 @@
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-apicurio-registry-protobuf-deployment</artifactId>
<version>${project.version}</version>
<type>pom</type>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>*</groupId>
<artifactId>*</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-arc-deployment</artifactId>
Expand Down
1 change: 1 addition & 0 deletions extensions/schema-registry/apicurio/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,5 +18,6 @@
<module>common</module>
<module>avro</module>
<module>json-schema</module>
<module>protobuf</module>
</modules>
</project>
45 changes: 45 additions & 0 deletions extensions/schema-registry/apicurio/protobuf/deployment/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<parent>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-apicurio-registry-protobuf-parent</artifactId>
<version>999-SNAPSHOT</version>
</parent>

<artifactId>quarkus-apicurio-registry-protobuf-deployment</artifactId>
<name>Quarkus - Apicurio Registry - Protobuf - Deployment</name>

<dependencies>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-apicurio-registry-protobuf</artifactId>
</dependency>

<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-apicurio-registry-common-deployment</artifactId>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<annotationProcessorPaths>
<path>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-extension-processor</artifactId>
<version>${project.version}</version>
</path>
</annotationProcessorPaths>
</configuration>
</plugin>
</plugins>
</build>

</project>
48 changes: 48 additions & 0 deletions ...rc/main/java/io/quarkus/apicurio/registry/protobuf/ApicurioRegistryProtobufProcessor.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
package io.quarkus.apicurio.registry.protobuf;

import io.quarkus.bootstrap.classloading.QuarkusClassLoader;
import io.quarkus.deployment.Feature;
import io.quarkus.deployment.annotations.BuildProducer;
import io.quarkus.deployment.annotations.BuildStep;
import io.quarkus.deployment.builditem.ExtensionSslNativeSupportBuildItem;
import io.quarkus.deployment.builditem.FeatureBuildItem;
import io.quarkus.deployment.builditem.nativeimage.ReflectiveClassBuildItem;

public class ApicurioRegistryProtobufProcessor {
@BuildStep
FeatureBuildItem feature() {
return new FeatureBuildItem(Feature.APICURIO_REGISTRY_PROTOBUF);
}

@BuildStep
public void apicurioRegistryProtobuf(BuildProducer<ReflectiveClassBuildItem> reflectiveClass,
BuildProducer<ExtensionSslNativeSupportBuildItem> sslNativeSupport) {

reflectiveClass
.produce(ReflectiveClassBuildItem.builder("io.apicurio.registry.serde.protobug.ProtobufKafkaDeserializer",
"io.apicurio.registry.serde.protobuf.ProtobufKafkaSerializer").methods().build());

reflectiveClass.produce(ReflectiveClassBuildItem.builder("io.apicurio.registry.serde.strategy.SimpleTopicIdStrategy",
"io.apicurio.registry.serde.strategy.TopicIdStrategy").methods().fields()
.build());

reflectiveClass.produce(ReflectiveClassBuildItem.builder("io.apicurio.registry.serde.DefaultIdHandler",
"io.apicurio.registry.serde.Legacy4ByteIdHandler",
"io.apicurio.registry.serde.fallback.DefaultFallbackArtifactProvider",
"io.apicurio.registry.serde.headers.DefaultHeadersHandler").methods().fields()
.build());

String defaultSchemaResolver = "io.apicurio.registry.serde.DefaultSchemaResolver";
if (QuarkusClassLoader.isClassPresentAtRuntime(defaultSchemaResolver)) {
// Class not present after 2.2.0.Final
reflectiveClass.produce(ReflectiveClassBuildItem.builder(defaultSchemaResolver).methods()
.fields().build());
}
}

@BuildStep
ExtensionSslNativeSupportBuildItem enableSslInNative() {
return new ExtensionSslNativeSupportBuildItem(Feature.APICURIO_REGISTRY_PROTOBUF);
}

}
31 changes: 31 additions & 0 deletions extensions/schema-registry/apicurio/protobuf/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>quarkus-apicurio-registry-parent</artifactId>
<groupId>io.quarkus</groupId>
<version>999-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>

<modelVersion>4.0.0</modelVersion>
<artifactId>quarkus-apicurio-registry-protobuf-parent</artifactId>
<name>Quarkus - Apicurio Registry - Protobuf</name>
<packaging>pom</packaging>

<dependencyManagement>
<dependencies>
<dependency>
<artifactId>kotlinx-serialization-core-jvm</artifactId>
<groupId>org.jetbrains.kotlinx</groupId>
<version>1.6.1</version>
</dependency>
</dependencies>
</dependencyManagement>

<modules>
<module>deployment</module>
<module>runtime</module>
</modules>
</project>
73 changes: 73 additions & 0 deletions extensions/schema-registry/apicurio/protobuf/runtime/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<parent>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-apicurio-registry-protobuf-parent</artifactId>
<version>999-SNAPSHOT</version>
</parent>

<artifactId>quarkus-apicurio-registry-protobuf</artifactId>
<name>Quarkus - Apicurio Registry - Protobuf - Runtime</name>
<description>Use Apicurio as Protobuf schema registry</description>

<dependencies>
<dependency>
<groupId>io.apicurio</groupId>
<artifactId>apicurio-registry-serdes-protobuf-serde</artifactId>
<exclusions>
<exclusion>
<groupId>io.apicurio</groupId>
<artifactId>apicurio-common-rest-client-jdk</artifactId>
</exclusion>
<exclusion>
<artifactId>checker-qual</artifactId>
<groupId>org.checkerframework</groupId>
</exclusion>
<exclusion>
<artifactId>slf4j-jboss-logging</artifactId>
<groupId>org.jboss.slf4j</groupId>
</exclusion>
<exclusion>
<artifactId>okhttp</artifactId>
<groupId>com.squareup.okhttp3</groupId>
</exclusion>
</exclusions>
</dependency>

<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-apicurio-registry-common</artifactId>
</dependency>
</dependencies>

<build>
<!-- Mark this as a runtime dependency, so to make sure it's included on the final classpath during native-image -->
<plugins>
<plugin>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-extension-maven-plugin</artifactId>
<configuration>
<capabilities>
<provides>io.quarkus.apicurio.registry.protobuf</provides>
</capabilities>
</configuration>
</plugin>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<annotationProcessorPaths>
<path>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-extension-processor</artifactId>
<version>${project.version}</version>
</path>
</annotationProcessorPaths>
</configuration>
</plugin>
</plugins>
</build>
</project>
12 changes: 12 additions & 0 deletions ...ema-registry/apicurio/protobuf/runtime/src/main/resources/META-INF/quarkus-extension.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
artifact: ${project.groupId}:${project.artifactId}:${project.version}
name: "Apicurio Registry - Protobuf"
metadata:
keywords:
- "apicurio"
- "protobuf"
- "kafka"
guide: "https://quarkus.io/guides/kafka-schema-registry-protobuf"
categories:
- "serialization"
status: "draft"
2 changes: 1 addition & 1 deletion extensions/schema-registry/confluent/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
<dependency>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-scripting-compiler-embeddable</artifactId>
<version>1.9.22</version>
<version>1.6.0</version>
</dependency>
<dependency>
<groupId>org.json</groupId>
Expand Down
1 change: 0 additions & 1 deletion ...s/enforcer-rules/src/main/resources/enforcer-rules/quarkus-banned-dependencies-okhttp.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@
<!-- Exclude okhttp -->
<exclude>com.squareup.okhttp3:*</exclude>
<exclude>com.squareup.okhttp:*</exclude>
<exclude>com.squareup.okio:*</exclude>
</excludes>
</bannedDependencies>
</rules>
Expand Down
2 changes: 1 addition & 1 deletion integration-tests/kafka-json-schema-apicurio2/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
<dependency>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-scripting-compiler-embeddable</artifactId>
<version>1.9.22</version>
<version>1.6.0</version>
</dependency>
<dependency>
<groupId>org.json</groupId>
Expand Down
Loading