New component - Azure Data Explorer

[ag-ramachandran]

The purpose and use-cases of the new component

Azure Data Explorer Exporter

This exporter sends metrics, logs and trace data to Azure Data Explorer.

Example configuration for the component

Configuration

The following settings are required:

• cluster_uri (no default): The cluster name of the provisioned ADX cluster to ingest the data.
• application_id (no default): The client id to connect to the cluster and ingest data.
• application_key (no default): The cluster secret corresponding to the client id.
• tenant_id (no default): The tenant id where the application_id is referenced from.

The following settings can be optionally configured and have default values:

Note that the database , tables are expected to be created upfront before the exporter is in operation , the definition of these are in the section Database and Table definition scripts

• db_name (default = "oteldb"): The ADX database where the tables are present to ingest the data.

• metrics_table_name (default = OTELMetrics): The target table in the database db_name that stores exported metric data.

• logs_table_name (default = OTELLogs): The target table in the database db_name that stores exported logs data.

• traces_table_name (default = OTELLogs): The target table in the database db_name that stores exported traces data.

  Optionally the following table mappings can be specified if the data needs to be mapped to a different table on ADX. This uses json table mapping that can be used to map attributes to target tables

• metrics_table_name_mapping (optional, no default): The table mapping name to be used for the table db_name.metrics_table_name

• logs_table_name_mapping (optional, no default): The table mapping name to be used for the table db_name.logs_table_name

• traces_table_name_mapping (optional, no default): The table mapping name to be used for the table db_name.traces_table_name

• ingestion_type (default = queued): ADX ingest can happen in managed streaming or queued modes.

Note: Streaming ingestion has to be enabled on ADX [configure the ADX cluster] in case of streaming option

An example configuration is provided as follows:

exporters:
  azuredataexplorer:
    # Kusto cluster uri
    cluster_uri: "https://CLUSTER.kusto.windows.net"
    # Client Id
    application_id: "f80da32c-108c-415c-a19e-643f461a677a"
    # The client secret for the client
    application_key: "xx-xx-xx-xx"
    # The tenant
    tenant_id: "21ff9e36-fbaa-43c8-98ba-00431ea10bc3"
    # Database for the logs
    db_name: "oteldb"
    # Metric table name
    metrics_table_name: "OTELMetrics"
    # Log table name
    logs_table_name: "OTELLogs"
    # Traces table
    traces_table_name: "OTELTraces"
    # Metric table mapping name
    metrics_table_name_mapping: "otelmetrics_mapping"
    # Log table mapping name
    logs_table_name_mapping: "otellogs_mapping"
    # Traces mapping table
    traces_table_name_mapping: "oteltraces_mapping"
    # Type of ingestion managed or queued
    ingestion_type : "managed"

Telemetry data types supported

Traces , Metrics and Logs are supported

Is this a vendor-specific component? If so, are you proposing to contribute this as a representative of the vendor?

Yes. This interfaces with Azure Data Explorer , a database offering from Microsoft

Sponsor (Optional)

[mx-psi]

@ag-ramachandran are you a representative of Microsoft?

[ag-ramachandran]

Hi @mx-psi , I am a part of the Microsoft organization (Microsoft is my current employer). Is there something specific that may be needed with respect to this , please let me know

[mx-psi]

Nothing else needed on that front, just someone that volunteers to maintain this (which could be you?). Also, if you don't have a sponsor the next one on the list should be @codeboten

[ag-ramachandran]

Thank you , we could create a DL for maintaining the component. For now it could be me.

[ag-ramachandran]

Hello @codeboten, sorry for the mention in case you are busy! Would you be able to be the sponsor for this component? We could set up some discussions and take this component forward!

[codeboten]

Happy to sponsor 👍🏻

[ag-ramachandran]

Thanks a lot @codeboten, what do you suggest as next steps. We've been doing some dev around the exporter and are in the process of internal reviews

[ag-ramachandran]

Hello Alex! Is there a specific structure for the review since this is a new component?
We have largely followed the code structure for other exporters. If there is a specific sequence/structure to the PR's we can start with that.
@codeboten

[ag-ramachandran]

Hello @codeboten , we just opened a PR for this new component as well. Is there something as a next step we can do so that we can add this component in.

Thank you

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.