HDFS-16467. Ensure Protobuf generated headers are included first #4601

GauthamBanasandra · 2022-07-20T17:12:10Z

This PR ensures that the Protobuf generated headers
are always included first, even when these headers
are included transitively.
This problem is specific to Windows only.

Description of PR

We need to ensure that the Protobuf generated headers (such as ClientNamenodeProtocol.pb.h) are included at the top. In other words, *.ph.h should be the first header files to be included in any of the .c/.cc/.h files. Otherwise, it causes symbol redefinition errors during compilation. Also, we need to ensure that Protobuf generated header files are the first ones to be included even in the case of transitive inclusion of header files.

This issue seems to be specific to Windows only. Not sure why the other platforms aren't running into this.

How was this patch tested?

Tested this locally by building on my Windows system.
Hadoop Jenkins CI validation.

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

* This PR ensures that the Protobuf generated headers are always included first, even when these headers are included transitively. * This problem is specific to Windows only.

hadoop-yetus · 2022-07-21T20:06:26Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 38s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 7 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	39m 55s		trunk passed
+1 💚	compile	3m 59s		trunk passed
+1 💚	mvnsite	0m 50s		trunk passed
+1 💚	shadedclient	63m 48s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 25s		the patch passed
+1 💚	compile	3m 38s		the patch passed
+1 💚	cc	3m 38s		the patch passed
+1 💚	golang	3m 38s		the patch passed
+1 💚	javac	3m 38s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	0m 28s		the patch passed
+1 💚	shadedclient	18m 59s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	32m 41s		hadoop-hdfs-native-client in the patch passed.
+1 💚	asflicense	0m 54s		The patch does not generate ASF License warnings.
		124m 18s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4601/2/artifact/out/Dockerfile
GITHUB PR	#4601
Optional Tests	dupname asflicense compile cc mvnsite javac unit codespell detsecrets golang
uname	Linux abcc7e8d04f2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `6845f0c`
Default Java	Red Hat, Inc.-1.8.0_332-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4601/2/testReport/
Max. process+thread count	560 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4601/2/console
versions	git=2.9.5 maven=3.6.3
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-07-21T22:16:29Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	21m 36s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 7 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	22m 42s		trunk passed
+1 💚	compile	4m 10s		trunk passed
+1 💚	mvnsite	1m 4s		trunk passed
+1 💚	shadedclient	47m 29s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 32s		the patch passed
+1 💚	compile	3m 45s		the patch passed
+1 💚	cc	3m 45s		the patch passed
+1 💚	golang	3m 45s		the patch passed
+1 💚	javac	3m 45s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	0m 36s		the patch passed
+1 💚	shadedclient	19m 6s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	33m 11s		hadoop-hdfs-native-client in the patch passed.
+1 💚	asflicense	0m 57s		The patch does not generate ASF License warnings.
		129m 50s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4601/2/artifact/out/Dockerfile
GITHUB PR	#4601
Optional Tests	dupname asflicense compile cc mvnsite javac unit codespell detsecrets golang
uname	Linux 61d15e62e08c 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `6845f0c`
Default Java	Red Hat, Inc.-1.8.0_312-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4601/2/testReport/
Max. process+thread count	553 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4601/2/console
versions	git=2.27.0 maven=3.6.3
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-07-22T00:41:13Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	20m 54s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 7 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	25m 19s		trunk passed
+1 💚	compile	3m 33s		trunk passed
+1 💚	mvnsite	0m 45s		trunk passed
+1 💚	shadedclient	55m 44s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 24s		the patch passed
+1 💚	compile	3m 16s		the patch passed
+1 💚	cc	3m 16s		the patch passed
+1 💚	golang	3m 16s		the patch passed
+1 💚	javac	3m 16s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	0m 28s		the patch passed
+1 💚	shadedclient	26m 34s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	33m 51s		hadoop-hdfs-native-client in the patch passed.
+1 💚	asflicense	0m 44s		The patch does not generate ASF License warnings.
		144m 29s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4601/2/artifact/out/Dockerfile
GITHUB PR	#4601
Optional Tests	dupname asflicense compile cc mvnsite javac unit codespell detsecrets golang
uname	Linux 2e12dc8bfcea 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `6845f0c`
Default Java	Debian-11.0.15+10-post-Debian-1deb10u1
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4601/2/testReport/
modules	C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4601/2/console
versions	git=2.20.1 maven=3.6.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-07-22T02:43:25Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 43s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 7 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	23m 19s		trunk passed
+1 💚	compile	4m 21s		trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	compile	4m 19s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	mvnsite	0m 39s		trunk passed
+1 💚	shadedclient	53m 58s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 22s		the patch passed
+1 💚	compile	3m 53s		the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	cc	3m 53s		the patch passed
+1 💚	golang	3m 54s		the patch passed
+1 💚	javac	3m 53s		the patch passed
+1 💚	compile	4m 0s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	cc	4m 0s		the patch passed
+1 💚	golang	4m 0s		the patch passed
+1 💚	javac	4m 0s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	0m 24s		the patch passed
+1 💚	shadedclient	21m 8s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	34m 31s		hadoop-hdfs-native-client in the patch passed.
+1 💚	asflicense	0m 39s		The patch does not generate ASF License warnings.
		122m 1s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4601/2/artifact/out/Dockerfile
GITHUB PR	#4601
Optional Tests	dupname asflicense compile cc mvnsite javac unit codespell detsecrets golang
uname	Linux 4017d1e252ac 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `6845f0c`
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4601/2/testReport/
Max. process+thread count	726 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4601/2/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

…che#4601) * This PR ensures that the Protobuf generated headers are always included first, even when these headers are included transitively. * This problem is specific to Windows only.

HDFS-16467. Ensure Protobuf generated headers are included first

517433f

* This PR ensures that the Protobuf generated headers are always included first, even when these headers are included transitively. * This problem is specific to Windows only.

GauthamBanasandra added the HDFS label Jul 20, 2022

GauthamBanasandra self-assigned this Jul 20, 2022

This comment was marked as outdated.

Sign in to view

Add pwd.h for Linux

6845f0c

GauthamBanasandra marked this pull request as ready for review July 22, 2022 14:39

GauthamBanasandra requested a review from goiri July 22, 2022 14:39

goiri approved these changes Jul 22, 2022

View reviewed changes

GauthamBanasandra merged commit 7de9b5e into apache:trunk Jul 23, 2022

GauthamBanasandra deleted the rearrange-headers branch July 23, 2022 17:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HDFS-16467. Ensure Protobuf generated headers are included first #4601

HDFS-16467. Ensure Protobuf generated headers are included first #4601

GauthamBanasandra commented Jul 20, 2022

This comment was marked as outdated.

hadoop-yetus commented Jul 21, 2022

hadoop-yetus commented Jul 21, 2022

hadoop-yetus commented Jul 22, 2022

hadoop-yetus commented Jul 22, 2022

HDFS-16467. Ensure Protobuf generated headers are included first #4601

HDFS-16467. Ensure Protobuf generated headers are included first #4601

Conversation

GauthamBanasandra commented Jul 20, 2022

Description of PR

How was this patch tested?

For code changes:

This comment was marked as outdated.

hadoop-yetus commented Jul 21, 2022

hadoop-yetus commented Jul 21, 2022

hadoop-yetus commented Jul 22, 2022

hadoop-yetus commented Jul 22, 2022