chore: update otlp docs

GreptimeTeam · Nov 7, 2024 · e2a2569 · e2a2569
1 parent 801e807
commit e2a2569
Show file tree

Hide file tree

Showing 3 changed files with 81 additions and 83 deletions.
diff --git a/docs/db-cloud-shared/clients/otlp-logs-integration.md b/docs/db-cloud-shared/clients/otlp-logs-integration.md
@@ -1,11 +1,9 @@
 GreptimeDB is an observability backend to consume OpenTelemetry Logs natively via [OTLP/HTTP](https://opentelemetry.io/docs/specs/otlp/#otlphttp) protocol.
 
-### API
+#### API
 
 To send OpenTelemetry Logs to GreptimeDB through OpenTelemetry SDK libraries, use the following information:
 
-#### Logs
-
 * URL: `https://<host>/v1/otlp/v1/logs`
 * Headers:
   * `X-Greptime-DB-Name`: `<dbname>`
@@ -15,7 +13,7 @@ To send OpenTelemetry Logs to GreptimeDB through OpenTelemetry SDK libraries, us
   * `X-Greptime-Log-Pipeline-Name`: `<pipeline_name>` (optional) - The pipeline name to process the logs. If not provided, the extract keys will be used to process the logs.
   * `X-Greptime-Log-Pipeline-Version`: `<pipeline_version>` (optional) - The pipeline version to process the logs. If not provided, the latest version of the pipeline will be used.
 
-The request uses binary protobuf to encode the payload, so you need to use packages that support `HTTP/protobuf`. For example, in Node.js, you can use [`exporter-trace-otlp-proto`](https://www.npmjs.com/package/@opentelemetry/exporter-trace-otlp-proto); in Go, you can use [`go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp`](https://pkg.go.dev/go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp); in Java, you can use [`io.opentelemetry:opentelemetry-exporter-otlp`](https://mvnrepository.com/artifact/io.opentelemetry/opentelemetry-exporter-otlp); and in Python, you can use [`opentelemetry-exporter-otlp-proto-http`](https://pypi.org/project/opentelemetry-exporter-otlp-proto-http/).
+The request uses binary protobuf to encode the payload, so you need to use packages that support `HTTP/protobuf`.
 
 :::tip NOTE
 The package names may change according to OpenTelemetry, so we recommend that you refer to the official OpenTelemetry documentation for the most up-to-date information.

diff --git a/docs/db-cloud-shared/clients/otlp-metrics-integration.md b/docs/db-cloud-shared/clients/otlp-metrics-integration.md
@@ -1,11 +1,9 @@
 GreptimeDB is an observability backend to consume OpenTelemetry Metrics natively via [OTLP/HTTP](https://opentelemetry.io/docs/specs/otlp/#otlphttp) protocol.
 
-### API
+#### API
 
 To send OpenTelemetry Metrics to GreptimeDB through OpenTelemetry SDK libraries, use the following information:
 
-#### Metrics
-
 * URL: `https://<host>/v1/otlp/v1/metrics`
 * Headers:
   * `X-Greptime-DB-Name`: `<dbname>`

diff --git a/docs/user-guide/ingest-data/for-observerbility/opentelemetry.md b/docs/user-guide/ingest-data/for-observerbility/opentelemetry.md
@@ -2,7 +2,9 @@
 
 [OpenTelemetry](https://opentelemetry.io/) is a vendor-neutral open-source observability framework for instrumenting, generating, collecting, and exporting telemetry data such as traces, metrics, logs. The OpenTelemetry Protocol (OTLP) defines the encoding, transport, and delivery mechanism of telemetry data between telemetry sources, intermediate processes such as collectors and telemetry backends.
 
-## OTLP/HTTP
+## Metrics
+
+### OTLP/HTTP
 
 import Includeotlpmetrycsintegration from '../../../db-cloud-shared/clients/otlp-metrics-integration.md' 
 
@@ -88,97 +90,97 @@ The example codes above may be outdated according to OpenTelemetry. We recommend
 
 For more information on the example code, please refer to the official documentation for your preferred programming language.
 
-import Includeotlplogintegration from '../../../db-cloud-shared/clients/otlp-logs-integration.md' 
-
-<Includeotlplogintegration/>
+#### Data Model
 
-#### Example Code
+The OTLP metrics data model is mapped to the GreptimeDB data model according to the following rules:
 
-Here are some example codes about how to setup the request in different languages:
+- The name of the Metric will be used as the name of the GreptimeDB table, and the table will be automatically created if it does not exist.
+- All attributes, including resource attributes, scope attributes, and data point attributes, will be used as tag columns of the GreptimeDB table.
+- The timestamp of the data point will be used as the timestamp index of GreptimeDB, and the column name is `greptime_timestamp`.
+- The data of Gauge/Sum data types will be used as the field column of GreptimeDB, and the column name is `greptime_value`.
+- Each quantile of the Summary data type will be used as a separated data column of GreptimeDB, and the column name is `greptime_pxx`, where xx is the quantile, such as 90/99, etc.
+- Histogram and ExponentialHistogram are not supported yet, we may introduce the Histogram data type to natively support these two types in a later version.
 
-<Tabs>
+## Logs
 
-<TabItem value="TypeScript" label="TypeScript">
+### OTLP/HTTP
 
-```ts
-const auth = Buffer.from(`${username}:${password}`).toString('base64')
-const exporter = new OTLPMetricExporter({
-    url: `https://${dbHost}/v1/otlp/v1/logs`,
-    headers: {
-        Authorization: `Basic ${auth}`,
-        "X-Greptime-DB-Name": db,
-    },
-    timeoutMillis: 5000,
-})
-```
-
-</TabItem>
-
-<TabItem value="Go" label="Go">
+import Includeotlplogintegration from '../../../db-cloud-shared/clients/otlp-logs-integration.md' 
 
-```Go
-auth := base64.StdEncoding.EncodeToString([]byte(fmt.Sprintf("%s:%s", *username, *password)))
-exporter, err := otlpmetrichttp.New(
-    context.Background(),
-    otlpmetrichttp.WithEndpoint(*dbHost),
-    otlpmetrichttp.WithURLPath("/v1/otlp/v1/logs"),
-    otlpmetrichttp.WithHeaders(map[string]string{
-        "X-Greptime-DB-Name": *dbName,
-        "Authorization":      "Basic " + auth,
-    }),
-    otlpmetrichttp.WithTimeout(time.Second*5),
-)
-```
+<Includeotlplogintegration/>
 
-</TabItem>
+#### Example Code
 
-<TabItem value="Java" label="Java">
+Here are some example codes about how to use Grafana Alloy to send OpenTelemetry logs to GreptimeDB:
 
-```Java
-String endpoint = String.format("https://%s/v1/otlp/v1/logs", dbHost);
-String auth = username + ":" + password;
-String b64Auth = new String(Base64.getEncoder().encode(auth.getBytes()));
-OtlpHttpMetricExporter exporter = OtlpHttpMetricExporter.builder()
-                .setEndpoint(endpoint)
-                .addHeader("X-Greptime-DB-Name", db)
-                .addHeader("Authorization", String.format("Basic %s", b64Auth))
-                .setTimeout(Duration.ofSeconds(5))
-                .build();
 ```
-
-</TabItem>
-
-<TabItem value="Python" label="Python">
-
-```python
-auth = f"{username}:{password}"
-b64_auth = base64.b64encode(auth.encode()).decode("ascii")
-endpoint = f"https://{host}/v1/otlp/v1/logs"
-exporter = OTLPMetricExporter(
-    endpoint=endpoint,
-    headers={"Authorization": f"Basic {b64_auth}", "X-Greptime-DB-Name": db},
-    timeout=5)
+loki.source.file "greptime" {
+  targets = [
+    {__path__ = "/tmp/foo.txt"},
+  ]
+  forward_to = [otelcol.receiver.loki.greptime.receiver]
+}
+
+otelcol.receiver.loki "greptime" {
+  output {
+    logs = [otelcol.exporter.otlphttp.greptimedb_logs.input]
+  }
+}
+
+otelcol.auth.basic "credentials" {
+  username = "${GREPTIME_USERNAME}"
+  password = "${GREPTIME_PASSWORD}"
+}
+
+otelcol.exporter.otlphttp "greptimedb_logs" {
+  client {
+    endpoint = "${GREPTIME_SCHEME:=http}://${GREPTIME_HOST:=greptimedb}:${GREPTIME_PORT:=4000}/v1/otlp/"
+    headers  = {
+      "X-Greptime-DB-Name" = "${GREPTIME_DB:=public}",
+      "x-greptime-log-table-name" = "demo_logs",
+      "x-greptime-log-extract-keys" = "filename,log.file.name,loki.attribute.labels",
+    }
+    auth     = otelcol.auth.basic.credentials.handler
+  }
+}
 ```
 
-</TabItem>
-
-</Tabs>
-
-You can find executable demos on GitHub at the links: [Go](https://github.com/GreptimeCloudStarters/quick-start-go), [Java](https://github.com/GreptimeCloudStarters/quick-start-java), [Python](https://github.com/GreptimeCloudStarters/quick-start-python), and [Node.js](https://github.com/GreptimeCloudStarters/quick-start-node-js).
+This example listens for changes to the file and sends the latest values to GreptimeDB via the otlp protocol.
 
 :::tip NOTE
-The example codes above may be outdated according to OpenTelemetry. We recommend that you refer to the official OpenTelemetry documentation for the most up-to-date information.
+The example codes above may be outdated according to OpenTelemetry. We recommend that you refer to the official OpenTelemetry documentation And Grafana Alloy for the most up-to-date information.
 :::
 
 For more information on the example code, please refer to the official documentation for your preferred programming language.
 
-## Data Model
-
-The OTLP metrics data model is mapped to the GreptimeDB data model according to the following rules:
+#### Data Model
+
+The OTLP logs data model is mapped to the GreptimeDB data model according to the following rules:
+
+Default table schema:
+
+```sql
++-----------------------+---------------------+------+------+---------+---------------+
+| Column                | Type                | Key  | Null | Default | Semantic Type |
++-----------------------+---------------------+------+------+---------+---------------+
+| timestamp             | TimestampNanosecond | PRI  | NO   |         | TIMESTAMP     |
+| trace_id              | String              |      | YES  |         | FIELD         |
+| span_id               | String              |      | YES  |         | FIELD         |
+| severity_text         | String              |      | YES  |         | FIELD         |
+| severity_number       | Int32               |      | YES  |         | FIELD         |
+| body                  | String              |      | YES  |         | FIELD         |
+| log_attributes        | Json                |      | YES  |         | FIELD         |
+| trace_flags           | UInt32              |      | YES  |         | FIELD         |
+| scope_name            | String              | PRI  | YES  |         | TAG           |
+| scope_version         | String              |      | YES  |         | FIELD         |
+| scope_attributes      | Json                |      | YES  |         | FIELD         |
+| scope_schema_url      | String              |      | YES  |         | FIELD         |
+| resource_attributes   | Json                |      | YES  |         | FIELD         |
+| resource_schema_url   | String              |      | YES  |         | FIELD         |
++-----------------------+---------------------+------+------+---------+---------------+
+17 rows in set (0.00 sec)
+```
 
-- The name of the Metric will be used as the name of the GreptimeDB table, and the table will be automatically created if it does not exist.
-- All attributes, including resource attributes, scope attributes, and data point attributes, will be used as tag columns of the GreptimeDB table.
-- The timestamp of the data point will be used as the timestamp index of GreptimeDB, and the column name is `greptime_timestamp`.
-- The data of Gauge/Sum data types will be used as the field column of GreptimeDB, and the column name is `greptime_value`.
-- Each quantile of the Summary data type will be used as a separated data column of GreptimeDB, and the column name is `greptime_pxx`, where xx is the quantile, such as 90/99, etc.
-- Histogram and ExponentialHistogram are not supported yet, we may introduce the Histogram data type to natively support these two types in a later version.
+- You can use `x-greptime-log-table-name` to specify the table name to store the logs. If not provided, the default table name is `opentelemetry_logs`.
+- All attributes, including resource attributes, scope attributes, and logs attributes, will be used as json column of the GreptimeDB table.
+- The timestamp of the log will be used as the timestamp index of GreptimeDB, and the column name is `timestamp`. prefer to use `time_unix_nano` ad the timestamp column. if not provided, the `observed_time_unix_nano` will be used.