Skip to content

Add Apache Spark / Amazon EMR connector decision and runtime #25

Description

@intraq-dev-ai

Connector gap

WrenAI lists Apache Spark / Amazon EMR as supported data sources. IntraQ maps Spark-like dialect behavior to Databricks in some helpers, but does not have a standalone Spark/EMR connector.

Evidence

WrenAI docs/README list Apache Spark and Amazon EMR. IntraQ live SQL execution currently supports Databricks SQL, but not a generic Spark Thrift Server or EMR connector.

Acceptance criteria

  • Decide whether Spark/EMR is supported as a standalone connector or explicitly documented as unsupported.
  • If supported, add connection fields for Spark Thrift Server / EMR endpoint configuration.
  • Support discovery, query preview, SQL Editor execution, Analyzer, and Dashboard Builder.
  • Add tests for connection validation, limits, and Spark SQL dialect behavior.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:data-sourcesData source connectors, discovery, and query runtimesenhancementNew feature or requesthelp wantedExtra attention is neededpriority:mediumUseful but not urgenttype:connectorNew or improved data-source connector support

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions