Skip to content

HBASE-25482 Improve SimpleRegionNormalizer#getAverageRegionSizeMb #2858

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jan 19, 2021
Merged

HBASE-25482 Improve SimpleRegionNormalizer#getAverageRegionSizeMb #2858

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e22f8c4
HBASE-25482 Improve SimpleRegionNormalizer#getAverageRegionSizeMb
ZhaoBQ Jan 8, 2021
a28e4b0
The log level cannot be used as a judgment condition for obtaining th…
ZhaoBQ Jan 11, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 13 additions & 12 deletions ...erver/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -257,16 +257,13 @@ private double getAverageRegionSizeMb(final List<RegionInfo> tableRegions) {
throw new IllegalStateException(
"Cannot calculate average size of a table without any regions.");
}
final int regionCount = tableRegions.size();
final long totalSizeMb = tableRegions.stream()
.mapToLong(this::getRegionSizeMB)
.sum();
TableName table = tableRegions.get(0).getTable();
int targetRegionCount = -1;
long targetRegionSize = -1;
double avgRegionSize;
try {
TableDescriptor tableDescriptor = masterServices.getTableDescriptors().get(table);
if (tableDescriptor != null && LOG.isDebugEnabled()) {
if (tableDescriptor != null) {
targetRegionCount = tableDescriptor.getNormalizerTargetRegionCount();
targetRegionSize = tableDescriptor.getNormalizerTargetRegionSize();
LOG.debug("Table {} configured with target region count {}, target region size {}", table,
Expand All @@ -276,18 +273,22 @@ private double getAverageRegionSizeMb(final List<RegionInfo> tableRegions) {
LOG.warn("TableDescriptor for {} unavailable, table-level target region count and size"
+ " configurations cannot be considered.", table, e);
}

double avgRegionSize;
if (targetRegionSize > 0) {
avgRegionSize = targetRegionSize;
} else if (targetRegionCount > 0) {
avgRegionSize = totalSizeMb / (double) targetRegionCount;
} else {
avgRegionSize = totalSizeMb / (double) regionCount;
final int regionCount = tableRegions.size();
final long totalSizeMb = tableRegions.stream()
.mapToLong(this::getRegionSizeMB)
.sum();
Comment on lines +280 to +282
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, just to be sure on the goal here, this is just to avoid this loop always happen, even when totalSizeMb is not been used?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is origin goal. If we set NormalizerTargetRegionSize, we use NormalizerTargetRegionSize as avgRegionSize and return. But now I found another serious bug, NormalizerTargetRegionCount and NormalizerTargetRegionSize will never used when log level is not debug.

if (targetRegionCount > 0) {
avgRegionSize = totalSizeMb / (double) targetRegionCount;
} else {
avgRegionSize = totalSizeMb / (double) regionCount;
}
LOG.debug("Table {}, total aggregated regions size: {} and average region size {}", table,
totalSizeMb, avgRegionSize);
}

LOG.debug("Table {}, total aggregated regions size: {} and average region size {}", table,
totalSizeMb, avgRegionSize);
return avgRegionSize;
}

Expand Down