Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HADOOP-18487. Protobuf 2.5 removal part 2: stop exporting protobuf-2.5 (#6185) #6258

Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
HADOOP-18487. Protobuf 2.5 removal part 2: stop exporting protobuf-2.5 (
#6185)


Followup to the previous HADOOP-18487 patch: changes the scope of
protobuf-2.5 in hadoop-common and elsewhere from "compile" to "provided".

This means that protobuf-2.5 is
* No longer included in hadoop distributions
* No longer exported by hadoop common POM files
* No longer exported transitively by other hadoop modules.
* No longer listed in LICENSE-binary.

Contributed by Steve Loughran
steveloughran committed Nov 6, 2023
commit 0e6bd7edb00b5f6849630750a5160b0ff8856dcd
24 changes: 15 additions & 9 deletions BUILDING.txt
Original file line number Diff line number Diff line change
@@ -301,22 +301,28 @@ Controlling the redistribution of the protobuf-2.5 dependency
the Hadoop codebase; alongside the move to Protobuf 3.x a private successor
class, org.apache.hadoop.ipc.internal.ShadedProtobufHelper is now used.

The hadoop-common JAR still declares a dependency on protobuf-2.5, but this
is likely to change in the future. The maven scope of the dependency can be
set with the common.protobuf2.scope option.
It can be set to "provided" in a build:
-Dcommon.protobuf2.scope=provided
If this is done then protobuf-2.5.0.jar will no longer be exported as a dependency,
and will then be omitted from the share/hadoop/common/lib/ directory of
any Hadoop distribution built. Any application declaring a dependency on hadoop-commmon
will no longer get the dependency; if they need it then they must explicitly declare it:
The hadoop-common module no longer exports its compile-time dependency on
protobuf-2.5. Hadoop distributions no longer include it.
Any application declaring a dependency on hadoop-commmon will no longer get
the artifact added to their classpath.
If is still required, then they must explicitly declare it:

<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
<version>2.5.0</version>
</dependency>

In Hadoop builds the scope of the dependency can be set with the
option "common.protobuf2.scope".
This can be upgraded from "provided" to "compile" on the maven command line:

-Dcommon.protobuf2.scope=compile

If this is done then protobuf-2.5.0.jar will again be exported as a
hadoop-common dependency, and included in the share/hadoop/common/lib/
directory of any Hadoop distribution built.

----------------------------------------------------------------------------------
Building components separately

1 change: 0 additions & 1 deletion LICENSE-binary
Original file line number Diff line number Diff line change
@@ -397,7 +397,6 @@ hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/d3.v3.js
hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/d3-v4.1.1.min.js
leveldb v1.13

com.google.protobuf:protobuf-java:2.5.0
com.google.protobuf:protobuf-java:3.6.1
com.google.protobuf:protobuf-java:3.7.1
com.google.re2j:re2j:1.1
4 changes: 2 additions & 2 deletions hadoop-project/pom.xml
Original file line number Diff line number Diff line change
@@ -90,8 +90,8 @@
<!-- This is used in hadoop-common for compilation only -->
<protobuf.version>2.5.0</protobuf.version>
<!-- Protobuf scope in hadoop common -->
<!-- set to "provided" and protobuf2 will no longer be exported as a dependency -->
<common.protobuf2.scope>compile</common.protobuf2.scope>
<!-- set to "provided" so protobuf2 is no longer exported as a dependency -->
<common.protobuf2.scope>provided</common.protobuf2.scope>
<!-- Protobuf scope in other modules which explicitly import the libarary -->
<transient.protobuf2.scope>${common.protobuf2.scope}</transient.protobuf2.scope>
<!-- ProtocolBuffer version, actually used in Hadoop -->