HBASE-26405 IntegrationTestLoadSmallValues #3802

apurtell · 2021-10-27T17:19:30Z

This integration test emulates a use case that stores a lot of small values into a table that would likely be heavily indexed (ROW_INDEX_V1, small blocks, etc.), an application that crowdsources weather (temperature) observation data. This IT can be used to test and optimize compression settings for such cases. It comes with a companion utility, HFileBlockExtracter, which extracts block data from HFiles into a set of local files for use in training external compression dictionaries, perhaps with ZStandard's zstd utility.

Run like:

./bin/hbase org.apache.hadoop.hbase.test.IntegrationTestLoadSmallValues
   numRows numMappers outputDir

You can also split the Loader and Verify stages:

Load with:

./bin/hbase 'org.apache.hadoop.hbase.test.IntegrationTestLoadSmallValues$Loader'
   numRows numMappers outputDir

Verify with:

./bin/hbase 'org.apache.hadoop.hbase.test.IntegrationTestLoadSmallValues$Verify'
   outputDir

Use HFileExtractor like so:

./bin/hbase org.apache.hadoop.hbase.test.util.HFileExtractor
   options outputDir hfile_1 ... hfile_n

Where options are:

 -d : Width of generated file name for zero padding, default: 5 
 -n : Total number of blocks to extract, default: unlimited 
 -r | --random: Shuffle blocks and write them in randomized order

You might train ZStandard dictionaries on the extracted block files like so:
(Assumes outputDir given to HFileExtractor was 't'.)

$ zstd --train -o dict t/*

Or:

$ zstd --train-fastcover=k=32,d=6 -o dict t/*

This was used to test the changes on HBASE-26353.

Apache-HBase · 2021-10-27T17:52:50Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 42s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 46s	master passed
+1 💚	compile	0m 33s	master passed
+1 💚	shadedjars	8m 49s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 22s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 54s	the patch passed
+1 💚	compile	0m 33s	the patch passed
+1 💚	javac	0m 33s	the patch passed
+1 💚	shadedjars	8m 13s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 19s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 58s	hbase-it in the patch passed.
		32m 25s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#3802
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 7304e0bde1c4 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `45f76a4`
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/1/testReport/
Max. process+thread count	692 (vs. ulimit of 30000)
modules	C: hbase-it U: hbase-it
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-10-27T17:53:03Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	4m 4s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 13s	master passed
+1 💚	compile	0m 29s	master passed
+1 💚	shadedjars	8m 15s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 20s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 54s	the patch passed
+1 💚	compile	0m 28s	the patch passed
+1 💚	javac	0m 28s	the patch passed
+1 💚	shadedjars	8m 17s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 17s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 0s	hbase-it in the patch passed.
		32m 32s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#3802
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 56d5a18bf73a 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `45f76a4`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/1/testReport/
Max. process+thread count	691 (vs. ulimit of 30000)
modules	C: hbase-it U: hbase-it
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-10-27T18:08:11Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 6s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	5m 2s	master passed
+1 💚	compile	0m 49s	master passed
+1 💚	checkstyle	0m 21s	master passed
+1 💚	spotbugs	0m 41s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 37s	the patch passed
-1 ❌	compile	0m 45s	hbase-it in the patch failed.
-0 ⚠️	javac	0m 45s	hbase-it in the patch failed.
-0 ⚠️	checkstyle	0m 18s	hbase-it: The patch generated 6 new + 0 unchanged - 0 fixed = 6 total (was 0)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	23m 19s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚	spotbugs	0m 50s	the patch passed
		_ Other Tests _
-1 ❌	asflicense	0m 14s	The patch generated 1 ASF License warnings.
		47m 7s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#3802
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux a6a2e1193406 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `45f76a4`
Default Java	AdoptOpenJDK-1.8.0_282-b08
compile	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/1/artifact/yetus-general-check/output/patch-compile-hbase-it.txt
javac	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/1/artifact/yetus-general-check/output/patch-compile-hbase-it.txt
checkstyle	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-it.txt
asflicense	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/1/artifact/yetus-general-check/output/patch-asflicense-problems.txt
Max. process+thread count	86 (vs. ulimit of 30000)
modules	C: hbase-it U: hbase-it
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/1/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

apurtell · 2021-10-27T19:15:09Z

Oops, fixed missing header license issue

Apache-HBase · 2021-10-27T19:44:21Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 26s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 50s	master passed
+1 💚	compile	0m 28s	master passed
+1 💚	shadedjars	8m 17s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 17s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 50s	the patch passed
+1 💚	compile	0m 28s	the patch passed
+1 💚	javac	0m 28s	the patch passed
+1 💚	shadedjars	8m 17s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 17s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 59s	hbase-it in the patch passed.
		28m 26s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#3802
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux ea7be7f36eb1 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `45f76a4`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/2/testReport/
Max. process+thread count	681 (vs. ulimit of 30000)
modules	C: hbase-it U: hbase-it
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/2/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-10-27T19:48:25Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 11s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 45s	master passed
+1 💚	compile	0m 32s	master passed
+1 💚	shadedjars	8m 51s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 19s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 49s	the patch passed
+1 💚	compile	0m 35s	the patch passed
+1 💚	javac	0m 35s	the patch passed
+1 💚	shadedjars	8m 52s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 19s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 1s	hbase-it in the patch passed.
		32m 32s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#3802
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux d5ad826c08ec 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `45f76a4`
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/2/testReport/
Max. process+thread count	692 (vs. ulimit of 30000)
modules	C: hbase-it U: hbase-it
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/2/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-10-27T20:05:52Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 6s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	5m 2s	master passed
+1 💚	compile	0m 48s	master passed
+1 💚	checkstyle	0m 22s	master passed
+1 💚	spotbugs	0m 42s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 42s	the patch passed
-1 ❌	compile	0m 48s	hbase-it in the patch failed.
-0 ⚠️	javac	0m 48s	hbase-it in the patch failed.
-0 ⚠️	checkstyle	0m 19s	hbase-it: The patch generated 6 new + 0 unchanged - 0 fixed = 6 total (was 0)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	23m 42s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚	spotbugs	1m 5s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 18s	The patch does not generate ASF License warnings.
		49m 18s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#3802
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux f91f30e3e185 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `45f76a4`
Default Java	AdoptOpenJDK-1.8.0_282-b08
compile	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/2/artifact/yetus-general-check/output/patch-compile-hbase-it.txt
javac	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/2/artifact/yetus-general-check/output/patch-compile-hbase-it.txt
checkstyle	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-it.txt
Max. process+thread count	86 (vs. ulimit of 30000)
modules	C: hbase-it U: hbase-it
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/2/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-10-27T20:39:44Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 18s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 50s	master passed
+1 💚	compile	0m 35s	master passed
+1 💚	shadedjars	8m 50s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 20s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 45s	the patch passed
+1 💚	compile	0m 34s	the patch passed
+1 💚	javac	0m 34s	the patch passed
+1 💚	shadedjars	9m 0s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 18s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 0s	hbase-it in the patch passed.
		32m 45s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#3802
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 6fd797684d84 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `45f76a4`
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/3/testReport/
Max. process+thread count	677 (vs. ulimit of 30000)
modules	C: hbase-it U: hbase-it
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/3/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-10-27T20:47:54Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	2m 28s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	5m 36s	master passed
+1 💚	compile	0m 37s	master passed
+1 💚	shadedjars	10m 56s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 25s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	5m 24s	the patch passed
+1 💚	compile	0m 37s	the patch passed
+1 💚	javac	0m 37s	the patch passed
+1 💚	shadedjars	11m 43s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 20s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 18s	hbase-it in the patch passed.
		40m 47s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#3802
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux d89f17a3547f 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `45f76a4`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/3/testReport/
Max. process+thread count	642 (vs. ulimit of 30000)
modules	C: hbase-it U: hbase-it
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/3/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-10-27T20:57:19Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 22s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	5m 0s	master passed
+1 💚	compile	0m 59s	master passed
+1 💚	checkstyle	0m 23s	master passed
+1 💚	spotbugs	0m 43s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 42s	the patch passed
+1 💚	compile	0m 51s	the patch passed
-0 ⚠️	javac	0m 51s	hbase-it generated 1 new + 102 unchanged - 1 fixed = 103 total (was 103)
+1 💚	checkstyle	0m 18s	the patch passed
+1 💚	whitespace	0m 1s	The patch has no whitespace issues.
+1 💚	hadoopcheck	24m 38s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚	spotbugs	0m 57s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 13s	The patch does not generate ASF License warnings.
		49m 43s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#3802
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux 636eb032ee10 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `45f76a4`
Default Java	AdoptOpenJDK-1.8.0_282-b08
javac	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/3/artifact/yetus-general-check/output/diff-compile-javac-hbase-it.txt
Max. process+thread count	86 (vs. ulimit of 30000)
modules	C: hbase-it U: hbase-it
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/3/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache9 · 2021-11-27T10:11:26Z

So what does the tool actually verify? The correctness or the performance of a compression algorithm?

apurtell · 2021-12-09T01:12:04Z

This was used to simulate a use case with small integer values in order to prove a compression improvement.

The value here, as such, is the simulation of a use case with a potentially very large set of rows comprised of small integer values / small cells. We don't need to accept this into the suite if there isn't enough value there. I have no strong opinion either way.

Apache-HBase · 2022-01-27T14:28:12Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 4s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	4m 11s	master passed
+1 💚	compile	0m 44s	master passed
+1 💚	checkstyle	0m 19s	master passed
+1 💚	spotbugs	0m 38s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 52s	the patch passed
+1 💚	compile	0m 43s	the patch passed
-0 ⚠️	javac	0m 43s	hbase-it generated 1 new + 102 unchanged - 1 fixed = 103 total (was 103)
+1 💚	checkstyle	0m 18s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	19m 44s	Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚	spotbugs	0m 46s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 15s	The patch does not generate ASF License warnings.
		40m 39s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#3802
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux 1086e2e117cf 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `f6348d4`
Default Java	AdoptOpenJDK-1.8.0_282-b08
javac	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/1/artifact/yetus-general-check/output/diff-compile-javac-hbase-it.txt
Max. process+thread count	96 (vs. ulimit of 30000)
modules	C: hbase-it U: hbase-it
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/1/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-01-27T14:35:17Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 26s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	5m 12s	master passed
+1 💚	compile	0m 31s	master passed
+1 💚	shadedjars	10m 5s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 20s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 54s	the patch passed
+1 💚	compile	0m 32s	the patch passed
+1 💚	javac	0m 32s	the patch passed
+1 💚	shadedjars	9m 58s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 19s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 8s	hbase-it in the patch passed.
		35m 28s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#3802
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux f661cadc6783 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `f6348d4`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/1/testReport/
Max. process+thread count	644 (vs. ulimit of 30000)
modules	C: hbase-it U: hbase-it
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-01-27T15:14:24Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 33s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	5m 45s	master passed
+1 💚	compile	0m 38s	master passed
+1 💚	shadedjars	10m 10s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 22s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	5m 45s	the patch passed
+1 💚	compile	0m 38s	the patch passed
+1 💚	javac	0m 38s	the patch passed
+1 💚	shadedjars	10m 15s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 22s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 9s	hbase-it in the patch passed.
		37m 41s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#3802
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 77bf8b488256 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `f6348d4`
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/1/testReport/
Max. process+thread count	634 (vs. ulimit of 30000)
modules	C: hbase-it U: hbase-it
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-02-01T07:32:37Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 26s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 14s	master passed
+1 💚	compile	0m 28s	master passed
+1 💚	shadedjars	8m 25s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 20s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 56s	the patch passed
+1 💚	compile	0m 29s	the patch passed
+1 💚	javac	0m 29s	the patch passed
+1 💚	shadedjars	8m 24s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 18s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 1s	hbase-it in the patch passed.
		29m 2s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#3802
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 3f250dd2240e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `351caa5`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/1/testReport/
Max. process+thread count	684 (vs. ulimit of 30000)
modules	C: hbase-it U: hbase-it
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-02-01T07:34:48Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 2s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 55s	master passed
+1 💚	compile	0m 32s	master passed
+1 💚	shadedjars	8m 27s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 23s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 32s	the patch passed
+1 💚	compile	0m 32s	the patch passed
+1 💚	javac	0m 32s	the patch passed
+1 💚	shadedjars	8m 26s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 19s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 59s	hbase-it in the patch passed.
		31m 8s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#3802
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux d713c99ae379 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `351caa5`
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/1/testReport/
Max. process+thread count	680 (vs. ulimit of 30000)
modules	C: hbase-it U: hbase-it
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3802/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

HBASE-26405 IntegrationTestLoadSmallValues

43b8131

Fix missing license header in HFileBlockExtracter

9eb48f5

Fix checkstyle and errorprone findings

a166c27

sunhelly approved these changes Dec 13, 2021

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-26405 IntegrationTestLoadSmallValues #3802

HBASE-26405 IntegrationTestLoadSmallValues #3802

apurtell commented Oct 27, 2021 •

edited

Loading

Apache-HBase commented Oct 27, 2021

Apache-HBase commented Oct 27, 2021

Apache-HBase commented Oct 27, 2021

apurtell commented Oct 27, 2021

Apache-HBase commented Oct 27, 2021

Apache-HBase commented Oct 27, 2021

Apache-HBase commented Oct 27, 2021

Apache-HBase commented Oct 27, 2021

Apache-HBase commented Oct 27, 2021

Apache-HBase commented Oct 27, 2021

Apache9 commented Nov 27, 2021

apurtell commented Dec 9, 2021

Apache-HBase commented Jan 27, 2022

Apache-HBase commented Jan 27, 2022

Apache-HBase commented Jan 27, 2022

Apache-HBase commented Feb 1, 2022

Apache-HBase commented Feb 1, 2022

HBASE-26405 IntegrationTestLoadSmallValues #3802

Are you sure you want to change the base?

HBASE-26405 IntegrationTestLoadSmallValues #3802

Conversation

apurtell commented Oct 27, 2021 • edited Loading

Apache-HBase commented Oct 27, 2021

Apache-HBase commented Oct 27, 2021

Apache-HBase commented Oct 27, 2021

apurtell commented Oct 27, 2021

Apache-HBase commented Oct 27, 2021

Apache-HBase commented Oct 27, 2021

Apache-HBase commented Oct 27, 2021

Apache-HBase commented Oct 27, 2021

Apache-HBase commented Oct 27, 2021

Apache-HBase commented Oct 27, 2021

Apache9 commented Nov 27, 2021

apurtell commented Dec 9, 2021

Apache-HBase commented Jan 27, 2022

Apache-HBase commented Jan 27, 2022

Apache-HBase commented Jan 27, 2022

Apache-HBase commented Feb 1, 2022

Apache-HBase commented Feb 1, 2022

apurtell commented Oct 27, 2021 •

edited

Loading