DRILL-8528: HBase Limit Push Down

[shfshihuafeng]

DRILL-8528: HBase Limit Push Down

Description

support Limit Push Down for HBase.i test

select * from clicks limit 3

The log shows that The storage layer hbase （HBaseRecordReader）only got 3 rows of data

2025-07-11 01:48:17,297 [178f302e-26db-37dd-500d-db2797b17a5c:frag:0:0] INFO o.a.d.e.s.hbase.HBaseRecordReader - Took 7 ms to get 3 records

plan is as follow

00-00    Screen : rowType = RecordType(ANY row_key, (VARCHAR(65535), ANY) MAP clickinfo, (VARCHAR(65535), ANY) MAP iteminfo): rowcount = 3.0, cumulative cost = {18.3 rows, 51.3 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 301
00-01      Project(row_key=[$0], clickinfo=[$1], iteminfo=[$2]) : rowType = RecordType(ANY row_key, (VARCHAR(65535), ANY) MAP clickinfo, (VARCHAR(65535), ANY) MAP iteminfo): rowcount = 3.0, cumulative cost = {18.0 rows, 51.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 300
00-02        SelectionVectorRemover : rowType = RecordType(ANY row_key, (VARCHAR(65535), ANY) MAP clickinfo, (VARCHAR(65535), ANY) MAP iteminfo): rowcount = 3.0, cumulative cost = {15.0 rows, 42.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 297
00-03          Limit(fetch=[3]) : rowType = RecordType(ANY row_key, (VARCHAR(65535), ANY) MAP clickinfo, (VARCHAR(65535), ANY) MAP iteminfo): rowcount = 3.0, cumulative cost = {12.0 rows, 39.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 296
00-04            Scan(table=[[hbase, clicks]], groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec [tableName="clicks"], columns=[`row_key`, `clickinfo`, `iteminfo`], maxRecords=3]]) : rowType = RecordType(ANY row_key, (VARCHAR(65535), ANY) MAP clickinfo, (VARCHAR(65535), ANY) MAP iteminfo): rowcount = 9.0, cumulative cost = {9.0 rows, 27.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 295

Documentation

(Please describe user-visible changes similar to what should appear in the Drill documentation.)

Testing

add test: org.apache.drill.hbase.TestHBaseFilterPushDown#testLimitPushDown

[cgivre]

@shfshihuafeng
Thanks for this. I have a few minor nits but other than that it looks good.

[cgivre]

Again is there a reason for the JsonIgnore?

[shfshihuafeng]

I don't think this is a necessary parameter to display in JSON, Do you suggest removing the annotation？

[cgivre]

It looks like I spoke too soon. The tests are failing for Java 17.

[shfshihuafeng]

It looks like I spoke too soon. The tests are failing for Java 17.
@cgivre the test cases passed In my local environment for java 17,May be trigger bugs that are difficult to troubleshoot, I will reproduce this issue

[shfshihuafeng]

It looks like I spoke too soon. The tests are failing for Java 17.

@cgivre The error is not caused by submitted code (hbase limit push down). I added my unit test cases to the original code( there is no hbase limit push down function). The plan shows that the limit has not been pushed down. but it still reports the same error which occurs occasionally.

[shfshihuafeng]

It looks like I spoke too soon. The tests are failing for Java 17.

@cgivre Maybe it would be best to open a new JIRA for this error and add some unit test for offset

[cgivre]

It looks like I spoke too soon. The tests are failing for Java 17.

@cgivre Maybe it would be best to open a new JIRA for this error and add some unit test for offset

I'm going to rerun the failing tests and see if that fixes it. Sometimes these tests are flaky and will randomly fail.

[cgivre]

LGTM +1. Thanks for submitting!