HBASE-27805 The chunk created by mslab may cause memory fragement and… #5193

bsglz · 2023-04-20T12:56:56Z

… lead to fullgc

Apache-HBase · 2023-04-20T14:12:26Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	2m 12s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for branch
+1 💚	mvninstall	3m 31s	master passed
+1 💚	compile	4m 43s	master passed
+1 💚	checkstyle	1m 9s	master passed
+0 🆗	refguide	2m 44s	branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚	spotless	0m 43s	branch has no errors when running spotless:check.
+1 💚	spotbugs	9m 3s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for patch
+1 💚	mvninstall	3m 21s	the patch passed
+1 💚	compile	4m 46s	the patch passed
+1 💚	javac	4m 46s	the patch passed
+1 💚	checkstyle	1m 4s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	xml	0m 1s	The patch has no ill-formed XML file.
+0 🆗	refguide	2m 15s	patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚	hadoopcheck	12m 41s	Patch does not cause any errors with Hadoop 3.2.4 3.3.4.
+1 💚	spotless	0m 40s	patch has no errors when running spotless:check.
+1 💚	spotbugs	9m 26s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 32s	The patch does not generate ASF License warnings.
		66m 45s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5193
Optional Tests	dupname asflicense javac refguide spotless xml spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux c7b560d5c812 5.4.0-144-generic #161-Ubuntu SMP Fri Feb 3 14:49:04 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `f5ee958`
Default Java	Eclipse Adoptium-11.0.17+8
refguide	https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-5193/1/yetus-general-check/output/branch-site/book.html
refguide	https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-5193/1/yetus-general-check/output/patch-site/book.html
Max. process+thread count	176 (vs. ulimit of 30000)
modules	C: hbase-common hbase-server . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/1/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-04-20T18:10:05Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	2m 25s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for branch
+1 💚	mvninstall	3m 30s	master passed
+1 💚	compile	1m 50s	master passed
+1 💚	shadedjars	4m 23s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	2m 30s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for patch
+1 💚	mvninstall	3m 19s	the patch passed
+1 💚	compile	1m 52s	the patch passed
+1 💚	javac	1m 52s	the patch passed
+1 💚	shadedjars	4m 24s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	2m 32s	the patch passed
		_ Other Tests _
+1 💚	unit	271m 28s	root in the patch passed.
		304m 27s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5193
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux b5d47b5ab987 5.4.0-137-generic #154-Ubuntu SMP Thu Jan 5 17:03:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `f5ee958`
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/1/testReport/
Max. process+thread count	5151 (vs. ulimit of 30000)
modules	C: hbase-common hbase-server . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-04-20T19:09:51Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 24s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 12s	Maven dependency ordering for branch
+1 💚	mvninstall	3m 50s	master passed
+1 💚	compile	1m 36s	master passed
+1 💚	shadedjars	5m 14s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	2m 14s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 10s	Maven dependency ordering for patch
+1 💚	mvninstall	3m 23s	the patch passed
+1 💚	compile	1m 29s	the patch passed
+1 💚	javac	1m 29s	the patch passed
+1 💚	shadedjars	4m 36s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	1m 51s	the patch passed
		_ Other Tests _
-1 ❌	unit	334m 24s	root in the patch failed.
		364m 20s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5193
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux e00082fe5de5 5.4.0-1094-aws #102~18.04.1-Ubuntu SMP Tue Jan 10 21:07:03 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `f5ee958`
Default Java	Temurin-1.8.0_352-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-root.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/1/testReport/
Max. process+thread count	2277 (vs. ulimit of 30000)
modules	C: hbase-common hbase-server . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-04-24T04:49:29Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 25s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+0 🆗	mvndep	0m 12s	Maven dependency ordering for branch
+1 💚	mvninstall	3m 35s	master passed
+1 💚	compile	4m 30s	master passed
+1 💚	checkstyle	1m 11s	master passed
+0 🆗	refguide	2m 29s	branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚	spotless	0m 41s	branch has no errors when running spotless:check.
+1 💚	spotbugs	8m 42s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 10s	Maven dependency ordering for patch
+1 💚	mvninstall	3m 30s	the patch passed
+1 💚	compile	4m 38s	the patch passed
+1 💚	javac	4m 38s	the patch passed
+1 💚	checkstyle	1m 10s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	xml	0m 1s	The patch has no ill-formed XML file.
+0 🆗	refguide	1m 58s	patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚	hadoopcheck	13m 3s	Patch does not cause any errors with Hadoop 3.2.4 3.3.4.
+1 💚	spotless	0m 38s	patch has no errors when running spotless:check.
+1 💚	spotbugs	8m 50s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 24s	The patch does not generate ASF License warnings.
		63m 58s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5193
Optional Tests	dupname asflicense javac refguide spotless xml spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux dd353ace1e38 5.4.0-1094-aws #102~18.04.1-Ubuntu SMP Tue Jan 10 21:07:03 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `95f83cb`
Default Java	Eclipse Adoptium-11.0.17+8
refguide	https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-5193/2/yetus-general-check/output/branch-site/book.html
refguide	https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-5193/2/yetus-general-check/output/patch-site/book.html
Max. process+thread count	176 (vs. ulimit of 30000)
modules	C: hbase-common hbase-server . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/2/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-04-24T09:04:02Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 23s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for branch
+1 💚	mvninstall	3m 49s	master passed
+1 💚	compile	2m 19s	master passed
+1 💚	shadedjars	4m 52s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	2m 39s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for patch
+1 💚	mvninstall	4m 2s	the patch passed
+1 💚	compile	1m 53s	the patch passed
+1 💚	javac	1m 53s	the patch passed
+1 💚	shadedjars	4m 21s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	2m 32s	the patch passed
		_ Other Tests _
+1 💚	unit	285m 24s	root in the patch passed.
		318m 36s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5193
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux db7267f512f2 5.4.0-137-generic #154-Ubuntu SMP Thu Jan 5 17:03:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `95f83cb`
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/2/testReport/
Max. process+thread count	4611 (vs. ulimit of 30000)
modules	C: hbase-common hbase-server . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-04-24T10:44:33Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 21s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 16s	Maven dependency ordering for branch
+1 💚	mvninstall	3m 4s	master passed
+1 💚	compile	1m 28s	master passed
+1 💚	shadedjars	4m 38s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	1m 52s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 10s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 47s	the patch passed
+1 💚	compile	1m 27s	the patch passed
+1 💚	javac	1m 27s	the patch passed
+1 💚	shadedjars	4m 36s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	2m 36s	the patch passed
		_ Other Tests _
+1 💚	unit	390m 6s	root in the patch passed.
		418m 54s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5193
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 4c2712c26b3a 5.4.0-1094-aws #102~18.04.1-Ubuntu SMP Tue Jan 10 21:07:03 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `95f83cb`
Default Java	Temurin-1.8.0_352-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/2/testReport/
Max. process+thread count	4798 (vs. ulimit of 30000)
modules	C: hbase-common hbase-server . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache9

Oh, there is a problem that, what is the default value for G1HeapRegionSize? After googling, I think the default size is based on the heap size?

The G1 GC is a regionalized and generational garbage collector, which means that the Java object heap (heap) is divided into a number of equally sized regions. Upon startup, the Java Virtual Machine (JVM) sets the region size. The region sizes can vary from 1 MB to 32 MB depending on the heap size. The goal is to have no more than 2048 regions. The eden, survivor, and old generations are logical sets of these regions and are not contiguous.

For having 2048 regions, if we have 16GB heap size, it will have 8MB region size, then there is no problem for using 2MB chunk size. And if we have less than 8GB heap size, 2047KB will also make a lot of humongous objects...

So I think a better approach is to documentation this out? If you are using G1 GC, be careful about the chunk size and the heap region size...

bsglz · 2023-05-05T09:49:30Z

Oh, there is a problem that, what is the default value for G1HeapRegionSize? After googling, I think the default size is based on the heap size?

The G1 GC is a regionalized and generational garbage collector, which means that the Java object heap (heap) is divided into a number of equally sized regions. Upon startup, the Java Virtual Machine (JVM) sets the region size. The region sizes can vary from 1 MB to 32 MB depending on the heap size. The goal is to have no more than 2048 regions. The eden, survivor, and old generations are logical sets of these regions and are not contiguous.

For having 2048 regions, if we have 16GB heap size, it will have 8MB region size, then there is no problem for using 2MB chunk size. And if we have less than 8GB heap size, 2047KB will also make a lot of humongous objects...

So I think a better approach is to documentation this out? If you are using G1 GC, be careful about the chunk size and the heap region size...

The G1HeapRegionSize calculated by initial_heap_size and max_heap_size, here are some cases for better understand:
//xms=0,xmx=10g -> region size 2048K
//xms=10g,xmx=10g -> region size 4096K
//xms=0,xmx=20g -> region size 4096K
//xms=20,xmx=20g -> region size 8192K
//xms=0,xmx=30g -> region size 4096K
//xms=0,xmx=32g -> region size 8192K

The related jdk8 code shows below:

void HeapRegion::setup_heap_region_size(size_t initial_heap_size, size_t max_heap_size) {
  uintx region_size = G1HeapRegionSize;
  if (FLAG_IS_DEFAULT(G1HeapRegionSize)) {
    size_t average_heap_size = (initial_heap_size + max_heap_size) / 2;
    region_size = MAX2(average_heap_size / HeapRegionBounds::target_number(),
                       (uintx) HeapRegionBounds::min_size());
  }

  int region_size_log = log2_long((jlong) region_size);
  // Recalculate the region size to make sure it's a power of
  // 2. This means that region_size is the largest power of 2 that's
  // <= what we've calculated so far.
  region_size = ((uintx)1 << region_size_log);

  // Now make sure that we don't go over or under our limits.
  if (region_size < HeapRegionBounds::min_size()) {
    region_size = HeapRegionBounds::min_size();
  } else if (region_size > HeapRegionBounds::max_size()) {
    region_size = HeapRegionBounds::max_size();
  }

  // And recalculate the log.
  region_size_log = log2_long((jlong) region_size);

  // Now, set up the globals.
  guarantee(LogOfHRGrainBytes == 0, "we should only set it once");
  LogOfHRGrainBytes = region_size_log;

  guarantee(LogOfHRGrainWords == 0, "we should only set it once");
  LogOfHRGrainWords = LogOfHRGrainBytes - LogHeapWordSize;

  guarantee(GrainBytes == 0, "we should only set it once");
  // The cast to int is safe, given that we've bounded region_size by
  // MIN_REGION_SIZE and MAX_REGION_SIZE.
  GrainBytes = (size_t)region_size;

  guarantee(GrainWords == 0, "we should only set it once");
  GrainWords = GrainBytes >> LogHeapWordSize;
  guarantee((size_t) 1 << LogOfHRGrainWords == GrainWords, "sanity");

  guarantee(CardsPerRegion == 0, "we should only set it once");
  CardsPerRegion = GrainBytes >> CardTableModRefBS::card_shift;
}

So it's easy to get 4M.

If the chunk size is 2047KB and the heapRegionSize is 2MB, we will just waste 1KB memory, its ok. But if the chunk size is 2048KB and the heapRegionSize is 4MB, we will waste 2MB memory.

Humongous objects are objects larger or equal the size of half a region. Every humongous object gets allocated as a sequence of contiguous regions in the old generation. The start of the object itself is always located at the start of the first region in that sequence. Any leftover space in the last region of the sequence will be lost for allocation until the entire object is reclaimed.

In practice, many users have little background knowledge about heapRegionSize and humongous objects and can easily run into problems, only document it seems not enough.

Apache9 · 2023-05-09T15:09:57Z

Let's post the new findings in the discussion thread in dev list to get more feedbacks? May also send to the user mailing list.

Changing the default may have big impact on performance so we should be careful...

bsglz · 2023-05-10T02:21:53Z

Let's post the new findings in the discussion thread in dev list to get more feedbacks? May also send to the user mailing list.

Changing the default may have big impact on performance so we should be careful...

Yeah, it really needs to be.

bbeaudreault · 2023-07-22T19:59:34Z

Personally I'm for documenting more than changing. If we need to make a change I think we should try to detect g1 region size. Otherwise leave it alone imo. It makes too much assumptions about user deployment to change default like this.

For us, we don't have this problem. Our region sizes are larger.

bsglz · 2023-07-23T02:21:36Z

Personally I'm for documenting more than changing. If we need to make a change I think we should try to detect g1 region size. Otherwise leave it alone imo. It makes too much assumptions about user deployment to change default like this.

For us, we don't have this problem. Our region sizes are larger.

Ok, let's document it.
Thanks for the comment.

bsglz · 2023-07-23T04:13:23Z

Ping. @Apache9 @bbeaudreault

Apache-HBase · 2023-07-23T05:20:53Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 28s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+0 🆗	mvndep	0m 14s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 55s	master passed
+1 💚	compile	4m 46s	master passed
+1 💚	checkstyle	1m 5s	master passed
+0 🆗	refguide	2m 36s	branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚	spotless	0m 45s	branch has no errors when running spotless:check.
+1 💚	spotbugs	8m 51s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 10s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 43s	the patch passed
+1 💚	compile	4m 42s	the patch passed
+1 💚	javac	4m 42s	the patch passed
+1 💚	checkstyle	1m 9s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	xml	0m 1s	The patch has no ill-formed XML file.
+0 🆗	refguide	2m 3s	patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚	hadoopcheck	10m 21s	Patch does not cause any errors with Hadoop 3.2.4 3.3.5.
+1 💚	spotless	0m 40s	patch has no errors when running spotless:check.
+1 💚	spotbugs	9m 20s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 25s	The patch does not generate ASF License warnings.
		60m 0s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5193
Optional Tests	dupname asflicense javac refguide spotless xml spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux 02e6078b28a9 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `73ea43f`
Default Java	Eclipse Adoptium-11.0.17+8
refguide	https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-5193/3/yetus-general-check/output/branch-site/book.html
refguide	https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-5193/3/yetus-general-check/output/patch-site/book.html
Max. process+thread count	176 (vs. ulimit of 30000)
modules	C: hbase-common hbase-server . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/3/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-07-23T09:31:28Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 14s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 47s	master passed
+1 💚	compile	1m 54s	master passed
+1 💚	shadedjars	4m 38s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	2m 31s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 33s	the patch passed
+1 💚	compile	1m 51s	the patch passed
+1 💚	javac	1m 51s	the patch passed
+1 💚	shadedjars	4m 36s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	2m 31s	the patch passed
		_ Other Tests _
+1 💚	unit	280m 51s	root in the patch passed.
		310m 42s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5193
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux a2e0cd408abd 5.4.0-148-generic #165-Ubuntu SMP Tue Apr 18 08:53:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `73ea43f`
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/3/testReport/
Max. process+thread count	8084 (vs. ulimit of 30000)
modules	C: hbase-common hbase-server . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/3/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-07-23T10:37:30Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 24s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 16s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 28s	master passed
+1 💚	compile	1m 25s	master passed
+1 💚	shadedjars	4m 46s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	1m 50s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 9s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 12s	the patch passed
+1 💚	compile	1m 29s	the patch passed
+1 💚	javac	1m 29s	the patch passed
+1 💚	shadedjars	4m 47s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	1m 50s	the patch passed
		_ Other Tests _
-1 ❌	unit	350m 0s	root in the patch failed.
		376m 42s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5193
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 8adbf03f4539 5.4.0-1101-aws #109~18.04.1-Ubuntu SMP Mon Apr 24 20:40:49 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `73ea43f`
Default Java	Temurin-1.8.0_352-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-root.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/3/testReport/
Max. process+thread count	5154 (vs. ulimit of 30000)
modules	C: hbase-common hbase-server . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/3/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-07-23T13:19:39Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 28s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+0 🆗	mvndep	0m 10s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 41s	master passed
+1 💚	compile	4m 46s	master passed
+1 💚	checkstyle	1m 5s	master passed
+0 🆗	refguide	2m 6s	branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚	spotless	0m 40s	branch has no errors when running spotless:check.
+1 💚	spotbugs	8m 54s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 10s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 42s	the patch passed
+1 💚	compile	4m 47s	the patch passed
+1 💚	javac	4m 47s	the patch passed
+1 💚	checkstyle	1m 3s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	xml	0m 1s	The patch has no ill-formed XML file.
+0 🆗	refguide	2m 1s	patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚	hadoopcheck	10m 2s	Patch does not cause any errors with Hadoop 3.2.4 3.3.5.
+1 💚	spotless	0m 41s	patch has no errors when running spotless:check.
+1 💚	spotbugs	9m 17s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 24s	The patch does not generate ASF License warnings.
		58m 29s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/4/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5193
Optional Tests	dupname asflicense javac refguide spotless xml spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux 29111dd02e59 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `73ea43f`
Default Java	Eclipse Adoptium-11.0.17+8
refguide	https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-5193/4/yetus-general-check/output/branch-site/book.html
refguide	https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-5193/4/yetus-general-check/output/patch-site/book.html
Max. process+thread count	177 (vs. ulimit of 30000)
modules	C: hbase-common hbase-server . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/4/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-07-23T16:49:21Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 13s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 10s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 45s	master passed
+1 💚	compile	1m 52s	master passed
+1 💚	shadedjars	4m 39s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	2m 34s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 30s	the patch passed
+1 💚	compile	1m 51s	the patch passed
+1 💚	javac	1m 51s	the patch passed
+1 💚	shadedjars	4m 36s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	2m 32s	the patch passed
		_ Other Tests _
-1 ❌	unit	239m 6s	root in the patch failed.
		268m 15s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5193
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 83246ff89a92 5.4.0-148-generic #165-Ubuntu SMP Tue Apr 18 08:53:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `73ea43f`
Default Java	Eclipse Adoptium-11.0.17+8
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/4/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-root.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/4/testReport/
Max. process+thread count	5370 (vs. ulimit of 30000)
modules	C: hbase-common hbase-server . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/4/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-07-23T19:30:03Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 24s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 15s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 16s	master passed
+1 💚	compile	1m 26s	master passed
+1 💚	shadedjars	4m 47s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	1m 50s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 10s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 14s	the patch passed
+1 💚	compile	1m 26s	the patch passed
+1 💚	javac	1m 26s	the patch passed
+1 💚	shadedjars	4m 45s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	1m 49s	the patch passed
		_ Other Tests _
+1 💚	unit	401m 30s	root in the patch passed.
		428m 48s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5193
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux e50b75a03f92 5.4.0-1101-aws #109~18.04.1-Ubuntu SMP Mon Apr 24 20:40:49 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `73ea43f`
Default Java	Temurin-1.8.0_352-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/4/testReport/
Max. process+thread count	8389 (vs. ulimit of 30000)
modules	C: hbase-common hbase-server . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/4/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

bsglz · 2023-07-24T02:45:19Z

This pr just update doc, the failure ut not related.

HBASE-27805 The chunk created by mslab may cause memory fragement and…

838c7d9

… lead to fullgc

Apache9 approved these changes Apr 28, 2023

View reviewed changes

Apache9 requested changes May 1, 2023

View reviewed changes

bsglz requested a review from Apache9 May 9, 2023 02:45

fix review issues

87e0692

Apache9 approved these changes Jul 24, 2023

View reviewed changes

bsglz merged commit cf81fd3 into apache:master Jul 27, 2023
1 check failed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-27805 The chunk created by mslab may cause memory fragement and… #5193

HBASE-27805 The chunk created by mslab may cause memory fragement and… #5193

bsglz commented Apr 20, 2023

Apache-HBase commented Apr 20, 2023

Apache-HBase commented Apr 20, 2023

Apache-HBase commented Apr 20, 2023

Apache-HBase commented Apr 24, 2023

Apache-HBase commented Apr 24, 2023

Apache-HBase commented Apr 24, 2023

Apache9 left a comment

bsglz commented May 5, 2023 •

edited

Loading

Apache9 commented May 9, 2023

bsglz commented May 10, 2023

bbeaudreault commented Jul 22, 2023

bsglz commented Jul 23, 2023

bsglz commented Jul 23, 2023

Apache-HBase commented Jul 23, 2023

Apache-HBase commented Jul 23, 2023

Apache-HBase commented Jul 23, 2023

Apache-HBase commented Jul 23, 2023

Apache-HBase commented Jul 23, 2023

Apache-HBase commented Jul 23, 2023

bsglz commented Jul 24, 2023

HBASE-27805 The chunk created by mslab may cause memory fragement and… #5193

HBASE-27805 The chunk created by mslab may cause memory fragement and… #5193

Conversation

bsglz commented Apr 20, 2023

Apache-HBase commented Apr 20, 2023

Apache-HBase commented Apr 20, 2023

Apache-HBase commented Apr 20, 2023

Apache-HBase commented Apr 24, 2023

Apache-HBase commented Apr 24, 2023

Apache-HBase commented Apr 24, 2023

Apache9 left a comment

Choose a reason for hiding this comment

bsglz commented May 5, 2023 • edited Loading

Apache9 commented May 9, 2023

bsglz commented May 10, 2023

bbeaudreault commented Jul 22, 2023

bsglz commented Jul 23, 2023

bsglz commented Jul 23, 2023

Apache-HBase commented Jul 23, 2023

Apache-HBase commented Jul 23, 2023

Apache-HBase commented Jul 23, 2023

Apache-HBase commented Jul 23, 2023

Apache-HBase commented Jul 23, 2023

Apache-HBase commented Jul 23, 2023

bsglz commented Jul 24, 2023

bsglz commented May 5, 2023 •

edited

Loading