Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

k8sobjects to elasticsearch failed for the field app #33801

Open
MaheshSankaran opened this issue Jun 28, 2024 · 6 comments
Open

k8sobjects to elasticsearch failed for the field app #33801

MaheshSankaran opened this issue Jun 28, 2024 · 6 comments
Labels

Comments

@MaheshSankaran
Copy link

Component(s)

receiver/k8sobjects

What happened?

Description

Trying to configure k8s object events to elasticsearch

Steps to Reproduce

receivers:
      k8sobjects:
        auth_type: serviceAccount
        objects:
          - name: events
            mode: watch
          - name: pods
            mode: pull


    exporters:
      logging:
        verbosity: detailed
      elasticsearch/events:
        endpoints: [https://elasticsearch-es-internal-http:9200]
        index: indexname
        user: elastic
        password: elastic
        sending_queue:
          enabled: true
          num_consumers: 1
          queue_size: 1000
        tls:
          insecure_skip_verify: true

    service:
      pipelines:
        logs:
          receivers:
          - k8sobjects
          exporters:
          - logging
          - elasticsearch/events

Expected Result

Actual Result

error elasticsearchexporter@v0.103.0/elasticsearch_bulk.go:309 Drop docs: failed to index: struct { Type string "json:"type""; Reason string "json:"reason"" }{Type:"document_parsing_exception", Reason:"[1:447] object mapping for [Body.metadata.labels.app] tried to parse field [app] as object, but found a concrete value"} {"kind": "exporter", "data_type": "logs", "name": "elasticsearch/events", "status": 400}
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter.(*worker).flush
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter@v0.103.0/elasticsearch_bulk.go:309
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter.(*worker).run
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter@v0.103.0/elasticsearch_bulk.go:291
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter.newBulkIndexer.func1
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter@v0.103.0/elasticsearch_bulk.go:207

Collector version

v0.103.0

Environment information

Environment

OS: (e.g., "Ubuntu 20.04")
Compiler(if manually compiled): (e.g., "go 14.2")

OpenTelemetry Collector configuration

receivers:
      k8sobjects:
        auth_type: serviceAccount
        objects:
          - name: events
            mode: watch
          - name: pods
            mode: pull


    exporters:
      logging:
        verbosity: detailed
      elasticsearch/events:
        endpoints: [https://elasticsearch-es-internal-http:9200]
        index: indexname
        user: elastic
        password: elastic
        sending_queue:
          enabled: true
          num_consumers: 1
          queue_size: 1000
        tls:
          insecure_skip_verify: true

    service:
      pipelines:
        logs:
          receivers:
          - k8sobjects
          exporters:
          - logging
          - elasticsearch/events

Log output

error	elasticsearchexporter@v0.103.0/elasticsearch_bulk.go:309	Drop docs: failed to index: struct { Type string "json:\"type\""; Reason string "json:\"reason\"" }{Type:"document_parsing_exception", Reason:"[1:447] object mapping for [Body.metadata.labels.app] tried to parse field [app] as object, but found a concrete value"}	{"kind": "exporter", "data_type": "logs", "name": "elasticsearch/events", "status": 400}
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter.(*worker).flush
	github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter@v0.103.0/elasticsearch_bulk.go:309
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter.(*worker).run
	github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter@v0.103.0/elasticsearch_bulk.go:291
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter.newBulkIndexer.func1
	github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter@v0.103.0/elasticsearch_bulk.go:207

Additional context

No response

@MaheshSankaran MaheshSankaran added bug Something isn't working needs triage New item requiring triage labels Jun 28, 2024
Copy link
Contributor

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

Copy link
Contributor

Pinging code owners for exporter/elasticsearch: @JaredTan95 @ycombinator @carsonip. See Adding Labels via Comments if you do not have permissions to add labels yourself.

@TylerHelmuth
Copy link
Member

This seems like a problem with elastic search translation not the k8s object receiver.

Copy link
Contributor

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions github-actions bot added the Stale label Aug 29, 2024
@JaredTan95 JaredTan95 removed the needs triage New item requiring triage label Aug 29, 2024
@JaredTan95
Copy link
Member

@MaheshSankaran hi, Can you desensitize the event data and put it here? So we can do a better analysis

@github-actions github-actions bot removed the Stale label Aug 30, 2024
Copy link
Contributor

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions github-actions bot added the Stale label Oct 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants