Skip to content

Commit

Permalink
Upgrade HBase community connector for Dremio 4.2.x
Browse files Browse the repository at this point in the history
- Bumped the dremio version to 4.2.x
- Upgraded protobuf and protobuf-maven-plugin versions
- Refactored some method signatures to ensure code compiles successfully.
  • Loading branch information
sbiswas authored and laurentgo committed May 7, 2020
1 parent 6147dbc commit 207c01a
Show file tree
Hide file tree
Showing 4 changed files with 30 additions and 6 deletions.
24 changes: 24 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,27 @@

[![Build Status](https://travis-ci.org/dremio-hub/dremio-hbase-connector.svg?branch=master)](https://travis-ci.org/dremio-hub/dremio-hbase-connector)

## Building and Installation

To add the Dremio Community HBase Connector to the list of Dremio sources:

1. From the project root, run `mvn clean install`.
The HBase Connector `dremio-hbase-plugin-4.2.2-202004211133290458-b550b6fa.jar` appears in the `target` directory.

1. Copy `dremio-hbase-plugin-4.2.2-202004211133290458-b550b6fa.jar` to the `<DREMIO_HOME>/jars` folder on all Dremio
nodes.

1. Run `<DREMIO_HOME>/bin/stop` to stop Dremio.

1. Download and copy the following jars to the `<DREMIO_HOME>/jars/3rdparty` folder on all Dremio nodes.
* [hbase-client-2.2.0.jar](https://repo1.maven.org/maven2/org/apache/hbase/hbase-client/2.2.0/hbase-client-2.2.0.jar)
* [hbase-common-2.2.0.jar](https://repo1.maven.org/maven2/org/apache/hbase/hbase-common/2.2.0/hbase-common-2.2.0.jar)
* [hbase-protocol-2.2.0.jar](https://repo1.maven.org/maven2/org/apache/hbase/hbase-protocol/2.2.0/hbase-protocol-2.2.0.jar)
* [hbase-protocol-shaded-2.2.0.jar](https://repo1.maven.org/maven2/org/apache/hbase/hbase-protocol-shaded/2.2.0/hbase-protocol-shaded-2.2.0.jar)
* [hbase-shaded-miscellaneous-2.2.0.jar](https://repo1.maven.org/maven2/org/apache/hbase/thirdparty/hbase-shaded-miscellaneous/2.2.0/hbase-shaded-miscellaneous-2.2.0.jar)
* [hbase-shaded-netty-2.2.0.jar](https://repo1.maven.org/maven2/org/apache/hbase/thirdparty/hbase-shaded-netty/2.2.0/hbase-shaded-netty-2.2.0.jar)
* [hbase-shaded-protobuf-2.2.0.jar](https://repo1.maven.org/maven2/org/apache/hbase/thirdparty/hbase-shaded-protobuf/2.2.0/hbase-shaded-protobuf-2.2.0.jar)

1. Run `<DREMIO_HOME>/bin/start` to start Dremio.

You now have the HBase Connector available in the list of sources on the Dremio UI.
8 changes: 4 additions & 4 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -20,14 +20,14 @@
<modelVersion>4.0.0</modelVersion>

<groupId>com.dremio.plugin</groupId>
<version>4.0.0-201909121834570395-c7a5071</version>
<version>4.2.2-202004211133290458-b550b6fa</version>
<artifactId>dremio-hbase-plugin</artifactId>

<name>Dremio HBase Community Connector</name>

<properties>
<hbase.TestSuite>**/HBaseTestsSuite.class</hbase.TestSuite>
<version.dremio>4.0.0-201909121834570395-c7a5071</version.dremio>
<version.dremio>4.2.2-202004211133290458-b550b6fa</version.dremio>
</properties>

<dependencies>
Expand Down Expand Up @@ -111,7 +111,7 @@
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
<version>2.5.0</version>
<version>3.9.1</version>
</dependency>

</dependencies>
Expand Down Expand Up @@ -142,7 +142,7 @@
<plugin>
<groupId>org.xolstice.maven.plugins</groupId>
<artifactId>protobuf-maven-plugin</artifactId>
<version>0.5.0</version>
<version>0.6.1</version>
<configuration>
<protoSourceRoot>${basedir}/src/main/proto</protoSourceRoot>
</configuration>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ public HBaseRecordReader(
}
}

public void setNumRowsPerBatch(long numRowsPerBatch) {
public void setNumRowsPerBatch(int numRowsPerBatch) {
this.numRowsPerBatch = numRowsPerBatch;
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ public HBaseScanDrel(RelOptCluster cluster, RelTraitSet traitSet, RelOptTable ta

@Override
public RelNode copy(RelTraitSet traitSet, List<RelNode> inputs) {
return new HBaseScanDrel(getCluster(), traitSet, table, pluginId, tableMetadata, projectedColumns, observedRowcountAdjustment);
return new HBaseScanDrel(getCluster(), traitSet, table, pluginId, tableMetadata, getProjectedColumns(), observedRowcountAdjustment);
}

@Override
Expand Down

0 comments on commit 207c01a

Please sign in to comment.