[Bug] NoClassDefFoundError: io/grpc/netty/shaded/io/grpc/netty/shaded/io/netty/handler/logging/ByteBufFormat #23513
Comments
|
@lhotari please take a look |
@suninuni Thanks for reporting this. Please help fixing this issue by providing exact steps to reproduce the problem. Providing all commands that need to be run will be a helpful first step. Thanks! Most likely this is caused by an issue in the shading configuration, but we need a good way to prevent future regressions. That could go in a integration / system test while the fix is made. |
lhotari
added
the
triage/lhotari/important
|
I think we need an integration test for Standalone's Related https://github.com/apache/pulsar/pull/17077/files This is a simple docker-compose.yml which can reproduce the problem |
|
This issue it also reproducible in pulsar 3.0.6: @lhotari pulsar/jetcd-core-shaded/pom.xml Lines 144 to 146 in 7822dca I’ve removed some configurations in my fork, and tests have passed with the This should address the issue, although the impact on IntelliJ support remains uncertain. |
|
Removing the attach and unpack configurations caused compile errors for some etcd-related classes in IntelliJ IDEA. IntelliJ expects the artifact JAR path to be target/{artifactId}-{version}.jar. However, the Pulsar build system sets finalName to Line 1762 in 81385c5 Changing finalName to |
|
I've raise a PR #23604 to address this problem, feel free to take a look, thanks. |
|
Thanks for the great work on addressing this issue, @Shawyeok! |
### Motivation There is a potential jar shading issue introduced in #4426 that causes `NoClassDefFoundError` when connecting to an etcd metadata store. The `jetcd-core-shaded` module was introduced in #4426 to address the compatibility issues between jetcd-core’s grpc-java dependency and Netty. You can find more details [here][1] and in the [grpc-java documentation][2]. [1]: #4426 (comment) [2]: https://github.com/grpc/grpc-java/blob/master/SECURITY.md#netty Currently, we use `unpack-shaded-jar` execution unpacks the shaded jar produced by `maven-shade-plugin:shade` into the `jetcd-core-shaded/target/classes` directory. However, the classes in this directory conflict with its dependencies. If the `maven-shade-plugin:shade` runs again without cleaning this directory, it can produce an incorrect shaded jar. You can replicate and verify this issue with the following commands: ```shell # Step 1: Clean the build directory mvn clean # Step 2: Perform an install and unpack the shaded jar into a directory. # Verify the import statement for `io.netty.handler.logging.ByteBufFormat` in # `org/apache/pulsar/jetcd/shaded/io/vertx/core/net/NetClientOptions.class`. # The correct import should be: # `import io.grpc.netty.shaded.io.netty.handler.logging.ByteBufFormat;`. mvn install unzip $M2_REPO/org/apache/bookkeeper/metadata/drivers/jetcd-core-shaded/4.18.0-SNAPSHOT/jetcd-core-shaded-4.18.0-SNAPSHOT-shaded.jar \ -d metadata-drivers/jetcd-core-shaded/target/first-classes # Step 3: Run the install command again without cleaning. # The unpacked jar from the previous step will persist in `target/classes`. # Unpack the shaded jar into a different directory (e.g., second-classes) and check the import. # The incorrect import will be: # `import io.grpc.netty.shaded.io.grpc.netty.shaded.io.netty.handler.logging.ByteBufFormat;`. mvn install unzip $M2_REPO/org/apache/bookkeeper/metadata/drivers/jetcd-core-shaded/4.18.0-SNAPSHOT/jetcd-core-shaded-4.18.0-SNAPSHOT-shaded.jar \ -d metadata-drivers/jetcd-core-shaded/target/second-classes # Step 4: Use IntelliJ IDEA's "Compare Directories" tool to compare the `first-classes` # and `second-classes` directories. The differences in imports should become apparent. ``` We can remove the attach and unpack configurations, and it should work fine. This issue has already been [reported][3] in apache/pulsar, and a similar [patch][patch] has addressed it. [3]: apache/pulsar#23513 [patch]: apache/pulsar#23604
Search before asking
Read release policy
Version
4.0.0
Minimal reproduce step
What did you expect to see?
no error.
What did you see instead?
The whole output for bookie-init job
Anything else?
Related #22892 (comment)
Are you willing to submit a PR?
The text was updated successfully, but these errors were encountered: