Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Webhookevent receiver timestamp not set #33702

Open
sdejong629 opened this issue Jun 21, 2024 · 4 comments
Open

Webhookevent receiver timestamp not set #33702

sdejong629 opened this issue Jun 21, 2024 · 4 comments
Labels

Comments

@sdejong629
Copy link

Component(s)

receiver/webhookevent

What happened?

Description

We export minio audit logs to an opentelemetry webhook_event_receiver, but no timestamp is set anywhere in the resulting logs. Clickhouse uses the timestamp to calculate the TTL of the log entries. The TTL expires immediately after the log has entered the system

Steps to Reproduce

  1. Configure a webhook event receiver in opentelemetry
  2. Configure for example minio auditing to send events to the local webhook event receiver: https://min.io/docs/minio/windows/operations/monitoring/minio-logging.html
  3. Export Log the file to a file and find no timestamp in the log entry
  4. Export the log to clickhouse and find that the timestamp is set to 1-1-1970

Expected Result

The expected result would be that a timestamp is set for at least the moment of receiving the event or maybe grab it from the minio logs itself (schema at https://min.io/docs/minio/windows/operations/monitoring/minio-logging.html#audit-log-structure) or some way to point the webhookeventreceiver where to get it's timestamp from.

Actual Result

No timestamp is set for a given log entry. Clickhouse for example therefor removes all logs after the TTL has expired (which is pretty much immediately)

Collector version

0.102.0

Environment information

Environment

Otel client on Ubuntu 22.04
Otel gateway on docker on Ubuntu 22.04

OpenTelemetry Collector configuration

receivers:
  webhookevent/minio_audit:
    endpoint: 0.0.0.0:9191
    read_timeout: "500ms"
    path: "/minio_audit/receiver"
    health_path: "/minio_audit/healthcheck"

processors:
  batch:

exporters:
  file/minio_audit:
    path: /data/logs/audit_logs.json
    rotation:
      max_backups: 10

service:
  extensions:
    - basicauth/logs
  pipelines:
    logs/minio_audit:
      receivers:
        - webhookevent/minio_audit
      processors:
        - batch
      exporters: [file/minio_audit]

Log output

{
  "resource": {
    "attributes": []
  },
  "scopeLogs": [
    {
      "scope": {
        "name": "otlp/webhookevent",
        "version": "0.102.0",
        "attributes": [
          {
            "key": "source",
            "value": {
              "stringValue": "webhookevent/minio_audit"
            }
          },
          {
            "key": "receiver",
            "value": {
              "stringValue": "webhookevent"
            }
          }
        ]
      },
      "logRecords": [
        {
          "body": {
            "stringValue": "REDACTED"
          },
          "traceId": "",
          "spanId": ""
        }
      ]
    }
  ],
  "schemaUrl": "https://opentelemetry.io/schemas/1.6.1"
}

Additional context

No response

@sdejong629 sdejong629 added bug Something isn't working needs triage New item requiring triage labels Jun 21, 2024
Copy link
Contributor

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@shalper2
Copy link
Contributor

Hey thanks for the issue! I think that there should be a timestamp set at the log level when the receiver gets an event (PR to follow). I would like to point out though that this value will have a different meaning than the time value in the event's payload.

@sdejong629
Copy link
Author

That's okay, at least this will be an indication of time at least

mx-psi pushed a commit that referenced this issue Aug 6, 2024
**Description:** Added the optional ObservedTimestamp field to log
record created when an event is received. Also fixed some erroneously
capitalized letters in two error descriptions.

**Link to tracking Issue:**
[33702](#33702)

**Testing:** built collector and passed unittests

**Documentation:** None
Copy link
Contributor

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions github-actions bot added the Stale label Aug 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants