Skip to content

HBASE-26762: Un-Deprecate and improve documentation for Scan#setRowPrefixFilter #4119

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Mar 2, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -95,9 +95,9 @@ public Scan withStopRow(byte[] stopRow, boolean inclusive) {
}

@Override
public Scan setRowPrefixFilter(byte[] rowPrefix) {
public Scan setStartStopRowForPrefixScan(byte[] rowPrefix) {
throw new UnsupportedOperationException(
"ImmutableScan does not allow access to setRowPrefixFilter");
"ImmutableScan does not allow access to setStartStopRowForPrefixScan");
}

@Override
Expand Down
47 changes: 37 additions & 10 deletions hbase-client/src/main/java/org/apache/hadoop/hbase/client/Scan.java
Original file line number Diff line number Diff line change
Expand Up @@ -339,6 +339,11 @@ public Scan setTimestamp(long timestamp) {
* <p>
* If the specified row does not exist, the Scanner will start from the next closest row after the
* specified row.
* <p>
* <b>Note:</b> <strong>Do NOT use this in combination with
* {@link #setRowPrefixFilter(byte[])} or {@link #setStartStopRowForPrefixScan(byte[])}.</strong>
* Doing so will make the scan result unexpected or even undefined.
* </p>
* @param startRow row to start scanner at or after
* @return this
* @throws IllegalArgumentException if startRow does not meet criteria for a row key (when length
Expand All @@ -354,7 +359,9 @@ public Scan withStartRow(byte[] startRow) {
* If the specified row does not exist, or the {@code inclusive} is {@code false}, the Scanner
* will start from the next closest row after the specified row.
* <p>
* <b>Note:</b> When use {@link #setRowPrefixFilter(byte[])}, the result might be unexpected.
* <b>Note:</b> <strong>Do NOT use this in combination with
* {@link #setRowPrefixFilter(byte[])} or {@link #setStartStopRowForPrefixScan(byte[])}.</strong>
* Doing so will make the scan result unexpected or even undefined.
* </p>
* @param startRow row to start scanner at or after
* @param inclusive whether we should include the start row when scan
Expand All @@ -377,8 +384,9 @@ public Scan withStartRow(byte[] startRow, boolean inclusive) {
* <p>
* The scan will include rows that are lexicographically less than the provided stopRow.
* <p>
* <b>Note:</b> When doing a filter for a rowKey <u>Prefix</u> use
* {@link #setRowPrefixFilter(byte[])}. The 'trailing 0' will not yield the desired result.
* <b>Note:</b> <strong>Do NOT use this in combination with
* {@link #setRowPrefixFilter(byte[])} or {@link #setStartStopRowForPrefixScan(byte[])}.</strong>
* Doing so will make the scan result unexpected or even undefined.
* </p>
* @param stopRow row to end at (exclusive)
* @return this
Expand All @@ -394,6 +402,11 @@ public Scan withStopRow(byte[] stopRow) {
* <p>
* The scan will include rows that are lexicographically less than (or equal to if
* {@code inclusive} is {@code true}) the provided stopRow.
* <p>
* <b>Note:</b> <strong>Do NOT use this in combination with
* {@link #setRowPrefixFilter(byte[])} or {@link #setStartStopRowForPrefixScan(byte[])}.</strong>
* Doing so will make the scan result unexpected or even undefined.
* </p>
* @param stopRow row to end at
* @param inclusive whether we should include the stop row when scan
* @return this
Expand All @@ -416,18 +429,32 @@ public Scan withStopRow(byte[] stopRow, boolean inclusive) {
* <p>This is a utility method that converts the desired rowPrefix into the appropriate values
* for the startRow and stopRow to achieve the desired result.</p>
* <p>This can safely be used in combination with setFilter.</p>
* <p><b>NOTE: Doing a {@link #withStartRow(byte[])} and/or {@link #withStopRow(byte[])}
* after this method will yield undefined results.</b></p>
* <p><strong>This CANNOT be used in combination with withStartRow and/or withStopRow.</strong>
* Such a combination will yield unexpected and even undefined results.</p>
* @param rowPrefix the prefix all rows must start with. (Set <i>null</i> to remove the filter.)
* @return this
* @deprecated since 3.0.0. The scan result might be unexpected in some cases.
* e.g. startRow : "112" and rowPrefixFilter : "11"
* The Result of this scan might contains : "111"
* This method implements the filter by setting startRow and stopRow,
* but does not take care of the scenario where startRow has been set.
* @deprecated since 2.5.0, will be removed in 4.0.0.
* The name of this method is considered to be confusing as it does not
* use a {@link Filter} but uses setting the startRow and stopRow instead.
* Use {@link #setStartStopRowForPrefixScan(byte[])} instead.
*/
@Deprecated
public Scan setRowPrefixFilter(byte[] rowPrefix) {
return setStartStopRowForPrefixScan(rowPrefix);
}

/**
* <p>Set a filter (using stopRow and startRow) so the result set only contains rows where the
* rowKey starts with the specified prefix.</p>
* <p>This is a utility method that converts the desired rowPrefix into the appropriate values
* for the startRow and stopRow to achieve the desired result.</p>
* <p>This can safely be used in combination with setFilter.</p>
* <p><strong>This CANNOT be used in combination with withStartRow and/or withStopRow.</strong>
* Such a combination will yield unexpected and even undefined results.</p>
* @param rowPrefix the prefix all rows must start with. (Set <i>null</i> to remove the filter.)
* @return this
*/
public Scan setStartStopRowForPrefixScan(byte[] rowPrefix) {
if (rowPrefix == null) {
withStartRow(HConstants.EMPTY_START_ROW);
withStopRow(HConstants.EMPTY_END_ROW);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -288,7 +288,7 @@ public static Scan makeQuotaSnapshotScanForTable(TableName tn) {
// Limit to "u:v" column
s.addColumn(QUOTA_FAMILY_USAGE, QUOTA_QUALIFIER_POLICY);
if (null == tn) {
s.setRowPrefixFilter(QUOTA_TABLE_ROW_KEY_PREFIX);
s.setStartStopRowForPrefixScan(QUOTA_TABLE_ROW_KEY_PREFIX);
} else {
byte[] row = getTableRowKey(tn);
// Limit rowspace to the "t:" prefix
Expand Down Expand Up @@ -637,7 +637,8 @@ static void deleteTableUsageSnapshotsForNamespace(Connection connection, String
throws IOException {
Scan s = new Scan();
//Get rows for all tables in namespace
s.setRowPrefixFilter(Bytes.add(QUOTA_TABLE_ROW_KEY_PREFIX, Bytes.toBytes(namespace + TableName.NAMESPACE_DELIM)));
s.setStartStopRowForPrefixScan(
Bytes.add(QUOTA_TABLE_ROW_KEY_PREFIX, Bytes.toBytes(namespace + TableName.NAMESPACE_DELIM)));
//Scan for table usage column (u:p) in quota table
s.addColumn(QUOTA_FAMILY_USAGE,QUOTA_QUALIFIER_POLICY);
//Scan for table quota column (q:s) if table has a space quota defined
Expand Down Expand Up @@ -706,7 +707,7 @@ static Scan createScanForNamespaceSnapshotSizes(String namespace) {
Scan s = new Scan();
if (namespace == null || namespace.isEmpty()) {
// Read all namespaces, just look at the row prefix
s.setRowPrefixFilter(QUOTA_NAMESPACE_ROW_KEY_PREFIX);
s.setStartStopRowForPrefixScan(QUOTA_NAMESPACE_ROW_KEY_PREFIX);
} else {
// Fetch the exact row for the table
byte[] rowkey = getNamespaceRowKey(namespace);
Expand All @@ -727,7 +728,7 @@ static Scan createScanForSpaceSnapshotSizes(TableName table) {
Scan s = new Scan();
if (null == table) {
// Read all tables, just look at the row prefix
s.setRowPrefixFilter(QUOTA_TABLE_ROW_KEY_PREFIX);
s.setStartStopRowForPrefixScan(QUOTA_TABLE_ROW_KEY_PREFIX);
} else {
// Fetch the exact row for the table
byte[] rowkey = getTableRowKey(table);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ public void testScanCopyConstructor() throws Exception {
.setReplicaId(3)
.setReversed(true)
.setRowOffsetPerColumnFamily(5)
.setRowPrefixFilter(Bytes.toBytes("row_"))
.setStartStopRowForPrefixScan(Bytes.toBytes("row_"))
.setScanMetricsEnabled(true)
.setReadType(Scan.ReadType.STREAM)
.withStartRow(Bytes.toBytes("row_1"))
Expand Down Expand Up @@ -181,10 +181,11 @@ private void testUnmodifiableSetters(Scan scanCopy) throws IOException {
assertEquals("ImmutableScan does not allow access to withStopRow", e.getMessage());
}
try {
scanCopy.setRowPrefixFilter(new byte[] { 1, 2 });
scanCopy.setStartStopRowForPrefixScan(new byte[] { 1, 2 });
throw new RuntimeException("Should not reach here");
} catch (UnsupportedOperationException e) {
assertEquals("ImmutableScan does not allow access to setRowPrefixFilter", e.getMessage());
assertEquals("ImmutableScan does not allow access to setStartStopRowForPrefixScan",
e.getMessage());
}
try {
scanCopy.readAllVersions();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -246,7 +246,7 @@ public void testScanCopyConstructor() throws Exception {
.setReplicaId(3)
.setReversed(true)
.setRowOffsetPerColumnFamily(5)
.setRowPrefixFilter(Bytes.toBytes("row_"))
.setStartStopRowForPrefixScan(Bytes.toBytes("row_"))
.setScanMetricsEnabled(true)
.setReadType(ReadType.STREAM)
.withStartRow(Bytes.toBytes("row_1"))
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@
import org.slf4j.LoggerFactory;

/**
* Test if Scan.setRowPrefixFilter works as intended.
* Test if Scan.setStartStopRowForPrefixScan works as intended.
*/
@Category({FilterTests.class, MediumTests.class})
public class TestScanRowPrefix extends FilterTestingCluster {
Expand All @@ -64,8 +64,8 @@ public void testPrefixScanning() throws IOException {
Table table = openTable(tableName);

/**
* Note that about half of these tests were relevant for an different implementation approach
* of setRowPrefixFilter. These test cases have been retained to ensure that also the
* Note that about half of these tests were relevant for a different implementation approach
* of setStartStopRowForPrefixScan. These test cases have been retained to ensure that also the
* edge cases found there are still covered.
*/

Expand Down Expand Up @@ -119,16 +119,16 @@ public void testPrefixScanning() throws IOException {
// ========
// PREFIX 0
Scan scan = new Scan();
scan.setRowPrefixFilter(prefix0);
scan.setStartStopRowForPrefixScan(prefix0);
verifyScanResult(table, scan, expected0, "Scan empty prefix failed");

// ========
// PREFIX 1
scan = new Scan();
scan.setRowPrefixFilter(prefix1);
scan.setStartStopRowForPrefixScan(prefix1);
verifyScanResult(table, scan, expected1, "Scan normal prefix failed");

scan.setRowPrefixFilter(null);
scan.setStartStopRowForPrefixScan(null);
verifyScanResult(table, scan, expected0, "Scan after prefix reset failed");

scan = new Scan();
Expand All @@ -138,10 +138,10 @@ public void testPrefixScanning() throws IOException {
// ========
// PREFIX 2
scan = new Scan();
scan.setRowPrefixFilter(prefix2);
scan.setStartStopRowForPrefixScan(prefix2);
verifyScanResult(table, scan, expected2, "Scan edge 0xFF prefix failed");

scan.setRowPrefixFilter(null);
scan.setStartStopRowForPrefixScan(null);
verifyScanResult(table, scan, expected0, "Scan after prefix reset failed");

scan = new Scan();
Expand All @@ -151,10 +151,10 @@ public void testPrefixScanning() throws IOException {
// ========
// PREFIX 3
scan = new Scan();
scan.setRowPrefixFilter(prefix3);
scan.setStartStopRowForPrefixScan(prefix3);
verifyScanResult(table, scan, expected3, "Scan normal with 0x00 ends failed");

scan.setRowPrefixFilter(null);
scan.setStartStopRowForPrefixScan(null);
verifyScanResult(table, scan, expected0, "Scan after prefix reset failed");

scan = new Scan();
Expand All @@ -164,10 +164,10 @@ public void testPrefixScanning() throws IOException {
// ========
// PREFIX 4
scan = new Scan();
scan.setRowPrefixFilter(prefix4);
scan.setStartStopRowForPrefixScan(prefix4);
verifyScanResult(table, scan, expected4, "Scan end prefix failed");

scan.setRowPrefixFilter(null);
scan.setStartStopRowForPrefixScan(null);
verifyScanResult(table, scan, expected0, "Scan after prefix reset failed");

scan = new Scan();
Expand All @@ -178,13 +178,13 @@ public void testPrefixScanning() throws IOException {
// COMBINED
// Prefix + Filter
scan = new Scan();
scan.setRowPrefixFilter(prefix1);
scan.setStartStopRowForPrefixScan(prefix1);
verifyScanResult(table, scan, expected1, "Prefix filter failed");

scan.setFilter(new ColumnPrefixFilter(prefix2));
verifyScanResult(table, scan, expected2, "Combined Prefix + Filter failed");

scan.setRowPrefixFilter(null);
scan.setStartStopRowForPrefixScan(null);
verifyScanResult(table, scan, expected2, "Combined Prefix + Filter; removing Prefix failed");

scan.setFilter(null);
Expand All @@ -196,55 +196,16 @@ public void testPrefixScanning() throws IOException {
scan.setFilter(new ColumnPrefixFilter(prefix2));
verifyScanResult(table, scan, expected2, "Test filter failed");

scan.setRowPrefixFilter(prefix1);
scan.setStartStopRowForPrefixScan(prefix1);
verifyScanResult(table, scan, expected2, "Combined Filter + Prefix failed");

scan.setFilter(null);
verifyScanResult(table, scan, expected1, "Combined Filter + Prefix ; removing Filter failed");

scan.setRowPrefixFilter(null);
scan.setStartStopRowForPrefixScan(null);
verifyScanResult(table, scan, expected0, "Scan after prefix reset failed");
}

@Test
public void testRowPrefixFilterAndStartRow() throws IOException {
final TableName tableName = TableName.valueOf(name.getMethodName());
createTable(tableName,"F");
Table table = openTable(tableName);

final byte[][] rowkeys = {Bytes.toBytes("111"), Bytes.toBytes("112")};
final byte[] prefixFilter = Bytes.toBytes("11");
for (byte[] rowkey: rowkeys) {
Put p = new Put(rowkey);
p.addColumn(Bytes.toBytes("F"), Bytes.toBytes("f"), Bytes.toBytes("test value"));
table.put(p);
}

List<byte[]> expected0 = new ArrayList<>();
expected0.add(rowkeys[0]);
expected0.add(rowkeys[1]);

List<byte[]> expected1 = new ArrayList<>();
expected1.add(rowkeys[1]);

// ========
// First scan
// Set startRow before setRowPrefixFilter
Scan scan = new Scan();
scan.withStartRow(rowkeys[1]);
scan.setRowPrefixFilter(prefixFilter);
verifyScanResult(table, scan, expected0, "Set startRow before setRowPrefixFilter unexpected");

// ========
// Second scan
// Set startRow after setRowPrefixFilter
// The result is different from first scan
scan = new Scan();
scan.setRowPrefixFilter(prefixFilter);
scan.withStartRow(rowkeys[1]);
verifyScanResult(table, scan, expected1, "Set startRow after setRowPrefixFilter unexpected");
}

private void verifyScanResult(Table table, Scan scan, List<byte[]> expectedKeys, String message) {
List<byte[]> actualKeys = new ArrayList<>();
try {
Expand Down
4 changes: 2 additions & 2 deletions hbase-shell/src/main/ruby/hbase/table.rb
Original file line number Diff line number Diff line change
Expand Up @@ -208,7 +208,7 @@ def _deleterows_internal(row, column = nil,

# create scan to get table names using prefix
scan = org.apache.hadoop.hbase.client.Scan.new
scan.setRowPrefixFilter(prefix.to_java_bytes)
scan.setStartStopRowForPrefixScan(prefix.to_java_bytes)
# Run the scanner to get all rowkeys
scanner = @table.getScanner(scan)
# Create a list to store all deletes
Expand Down Expand Up @@ -536,7 +536,7 @@ def _hash_to_scan(args)
end

# This will overwrite any startrow/stoprow settings
scan.setRowPrefixFilter(rowprefixfilter.to_java_bytes) if rowprefixfilter
scan.setStartStopRowForPrefixScan(rowprefixfilter.to_java_bytes) if rowprefixfilter

# Clear converters from last scan.
@converters.clear
Expand Down
2 changes: 1 addition & 1 deletion src/main/asciidoc/_chapters/datamodel.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -300,7 +300,7 @@ Table table = ... // instantiate a Table instance

Scan scan = new Scan();
scan.addColumn(CF, ATTR);
scan.setRowPrefixFilter(Bytes.toBytes("row"));
scan.setStartStopRowForPrefixScan(Bytes.toBytes("row"));
ResultScanner rs = table.getScanner(scan);
try {
for (Result r = rs.next(); r != null; r = rs.next()) {
Expand Down