Add new sink: Apache Pinot

[jasodeep]

Q: Can the proposed component’s functionality be replicated by an existing component, with a specific configuration ?
A: No.

Q: Alternatively implemented as a wrapper around an existing component. (ex. axiom wrapping elasticsearch)
A: No.

Q: Can an existing component replicate the proposed component’s functionality, with non-breaking changes?
A: No.

Q: Can an existing component be rewritten in a more generic fashion to cover both the existing and proposed functions?
A: No.

Q: Is the proposed component generically usable or is it specific to a particular service?
A: Can be generically usable to push data to apache pinot.

Q: Is there sufficient demand for the component?
A: Apache pinot is now being used heavily for OLAP(Online Analytical Processing) datastore.

Q: Is the contribution from an individual or the organization owning the integrated service? (examples of organization backed integrations: databend sink, axiom sink)
A: Individual

Q: Is the contributor committed to maintaining the integration if it is accepted?
A: Yes

Q: What is the overall complexity of the proposed design of this integration from a technical and functional standpoint, and what is the expected ongoing maintenance burden?
A: we need to keep the sink compatible with newer versions of Apache Pinot

Q: Will we have access to an account with the service if the integration is not open source?
A: Apache pinot is open source

Tasks

Beta Give feedback

No tasks being tracked yet.

[jasodeep]

@jszwedko are we open to contribution to add this new sink? I can start working on this issue.

[jszwedko]

Hi @jasodeep ,

Thanks for opening this request and filling out all of the details. One question I have is: how do plan to transmit data from Vector to Apache Pinot? I took a quick look at their docs and only saw mechanisms for Apache Pinot to ingest data from various sources, but not how to send data directly to it (e.g. via an HTTP API).