Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade Pinot Connector Libraries to 0.8.0 #9098

Merged
merged 5 commits into from
Nov 5, 2021
Merged

Upgrade Pinot Connector Libraries to 0.8.0 #9098

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
0eb49f2
Rename PinotQuery to PinotQueryInfo
elonazoulay Oct 17, 2021
2f0275e
Use JsonCodec when submitting broker request in Pinot
elonazoulay Oct 17, 2021
ad0a5d7
Remove PinotColumn
elonazoulay Oct 17, 2021
1fb2ff2
Upgrade libraries to pinot-0.8.0
elonazoulay Oct 17, 2021
f02d3fe
Fix offset handling in passthrough queries
elonazoulay Oct 27, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/src/main/sphinx/connector/pinot.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Requirements

To connect to Pinot, you need:

* Pinot 0.1.0 or higher.
* Pinot 0.8.0 or higher.
* Network access from the Trino coordinator and workers to the Pinot controller
nodes. Port 8098 is the default port.

Expand Down
45 changes: 38 additions & 7 deletions plugin/trino-pinot/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@

<properties>
<air.main.basedir>${project.parent.basedir}</air.main.basedir>
<dep.pinot.version>0.6.0</dep.pinot.version>
<dep.pinot.version>0.8.0</dep.pinot.version>
</properties>

<dependencyManagement>
Expand Down Expand Up @@ -103,12 +103,6 @@
<artifactId>guice</artifactId>
</dependency>

<dependency>
<groupId>com.yammer.metrics</groupId>
<artifactId>metrics-core</artifactId>
<version>2.2.0</version>
</dependency>

<dependency>
<groupId>commons-codec</groupId>
<artifactId>commons-codec</artifactId>
Expand Down Expand Up @@ -325,10 +319,18 @@
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-annotations</artifactId>
</exclusion>
<exclusion>
<groupId>jakarta.ws.rs</groupId>
<artifactId>jakarta.ws.rs-api</artifactId>
</exclusion>
<exclusion>
<groupId>javax.validation</groupId>
<artifactId>validation-api</artifactId>
</exclusion>
<exclusion>
<groupId>org.glassfish.hk2.external</groupId>
<artifactId>jakarta.inject</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.lucene</groupId>
<artifactId>lucene-analyzers-common</artifactId>
Expand All @@ -340,6 +342,28 @@
</exclusions>
</dependency>

<dependency>
<groupId>org.apache.pinot</groupId>
<artifactId>pinot-segment-local</artifactId>
<version>${dep.pinot.version}</version>
<exclusions>
<exclusion>
<groupId>org.apache.lucene</groupId>
<artifactId>lucene-analyzers-common</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.lucene</groupId>
<artifactId>lucene-core</artifactId>
</exclusion>
</exclusions>
</dependency>

<dependency>
<groupId>org.apache.pinot</groupId>
<artifactId>pinot-segment-spi</artifactId>
<version>${dep.pinot.version}</version>
</dependency>

<dependency>
<groupId>org.apache.pinot</groupId>
<artifactId>pinot-spi</artifactId>
Expand Down Expand Up @@ -382,6 +406,13 @@
<scope>runtime</scope>
</dependency>

<dependency>
<groupId>org.apache.pinot</groupId>
<artifactId>pinot-yammer</artifactId>
<version>${dep.pinot.version}</version>
<scope>runtime</scope>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how are we getting this during runtime?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shall we make a noop implementation for this pinot-metrics interface ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how are we getting this during runtime?

Required by the PinotQueryClient which requires PinotMetricsFactory. Would it be ok to leave it, so that we can have broker metrics? The worker functions like a broker without a reducer service (soon we will put that in also). wdyt?

</dependency>

<!-- Trino SPI -->
<dependency>
<groupId>io.trino</groupId>
Expand Down
6 changes: 3 additions & 3 deletions plugin/trino-pinot/src/main/java/io/trino/plugin/pinot/PinotBrokerPageSource.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
import io.trino.plugin.pinot.client.PinotClient.BrokerResultRow;
import io.trino.plugin.pinot.decoders.Decoder;
import io.trino.plugin.pinot.decoders.DecoderFactory;
import io.trino.plugin.pinot.query.PinotQuery;
import io.trino.plugin.pinot.query.PinotQueryInfo;
import io.trino.spi.Page;
import io.trino.spi.block.Block;
import io.trino.spi.block.BlockBuilder;
Expand All @@ -39,7 +39,7 @@
public class PinotBrokerPageSource
implements ConnectorPageSource
{
private final PinotQuery query;
private final PinotQueryInfo query;
private final PinotClient pinotClient;
private final ConnectorSession session;
private final List<PinotColumnHandle> columnHandles;
Expand All @@ -56,7 +56,7 @@ public class PinotBrokerPageSource

public PinotBrokerPageSource(
ConnectorSession session,
PinotQuery query,
PinotQueryInfo query,
List<PinotColumnHandle> columnHandles,
PinotClient pinotClient,
int limitForBrokerQueries)
Expand Down
135 changes: 0 additions & 135 deletions plugin/trino-pinot/src/main/java/io/trino/plugin/pinot/PinotColumn.java
View file
Open in desktop

This file was deleted.

Loading