Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Geoip processor not found #34737

Closed
grioda01 opened this issue Aug 19, 2024 · 6 comments
Closed

Geoip processor not found #34737

grioda01 opened this issue Aug 19, 2024 · 6 comments

Comments

@grioda01
Copy link

Component(s)

processor/geoip

What happened?

Description

I set up the opentelemetrycollector resource on an azure kubernetes cluster, built on a Opentelemetry helm operator version 0.67.0
the collector image used is otel/opentelemetry-collector-contrib:0.106.1

I am getting the following error from the opentelemetrycollector

Error: failed to get config: cannot unmarshal the configuration: decoding failed due to the following error(s):

error decoding 'processors': unknown type: "geoip" for id: "geoip" (valid values: [attributes cumulativetodelta probabilistic_sampler resource span tail_sampling memory_limiter deltatorate groupbyattrs metricstransform resourcedetection routing sumologic batch filter groupbytrace experimental_metricsgeneration redaction remotetap transform k8sattributes])

Steps to Reproduce

Intall Opentelemetry Operator helm chart
Install opentelemetrycollector resource with geoip included in processor of config

Expected Result

Geoip processor should be available

Actual Result

I am getting the following error from the opentelemetrycollector

Error: failed to get config: cannot unmarshal the configuration: decoding failed due to the following error(s):

error decoding 'processors': unknown type: "geoip" for id: "geoip" (valid values: [attributes cumulativetodelta probabilistic_sampler resource span tail_sampling memory_limiter deltatorate groupbyattrs metricstransform resourcedetection routing sumologic batch filter groupbytrace experimental_metricsgeneration redaction remotetap transform k8sattributes])

Collector version

0.106.1

Environment information

Environment

OS: (e.g., "Ubuntu 20.04")
Compiler(if manually compiled): (e.g., "go 14.2")
Containerized solution in Azure Kubernetes cluster, via Opentelemetry Operator Helm chart

OpenTelemetry Collector configuration

exporters:
      debug:
        verbosity: detailed
      elasticsearch:
        auth:
          authenticator: basicauth
        endpoint: xxxxxx
        logs_index: xxxxxxx
        pipeline: adjust_otel
    extensions:
      basicauth:
        client_auth:
          password: xxxxxxxx
          username: xxxxxxxxx
    processors:
      batch: {}
      geoip:
        providers:
          maxmind:
            database_path: /tmp/geoipdb/GeoLite2-City.mmdb
      k8sattributes:
        extract:
          labels:
          - from: node
            key: kubernetes.azure.com/cluster
            tag_name: k8s.cluster.name
          metadata:
          - k8s.pod.name
          - k8s.pod.uid
          - k8s.deployment.name
          - k8s.namespace.name
          - k8s.node.name
          - k8s.pod.start_time
          - k8s.cluster.uid
        pod_association:
        - sources:
          - from: resource_attribute
            name: k8s.pod.name
          - from: resource_attribute
            name: k8s.namespace.name
          - from: resource_attribute
            name: k8s.pod.uid
    receivers:
      filelog:
        exclude:
        - /var/log/pods/accurint-apps*/acccron/*.log
        - /var/log/pods/accurint-apps*/clamav/*.log
        include:
        - /var/log/pods/accurint-apps*/*/*.log
        - /var/log/pods/bwg-apps*/*/*.log
    service:
      extensions:
      - basicauth
      pipelines:
        logs:
          exporters:
          - debug
          - elasticsearch
          processors:
          - geoip
          - k8sattributes
          - batch
          receivers:
          - filelog

Log output

Error: failed to get config: cannot unmarshal the configuration: decoding failed due to the following error(s):

error decoding 'processors': unknown type: "geoip" for id: "geoip" (valid values: [attributes cumulativetodelta probabilistic_sampler resource span tail_sampling memory_limiter deltatorate groupbyattrs metricstransform resourcedetection routing sumologic batch filter groupbytrace experimental_metricsgeneration redaction remotetap transform k8sattributes])

Additional context

No response

@grioda01 grioda01 added bug Something isn't working needs triage New item requiring triage labels Aug 19, 2024
Copy link
Contributor

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@crobert-1
Copy link
Member

Hello @grioda01, this component is marked as Development stability, which means it's still in active development and not included in releases yet.

The code owners can share more information around plans to mark this component as alpha, and include it in future releases of the collector 👍

@crobert-1 crobert-1 removed bug Something isn't working needs triage New item requiring triage labels Aug 19, 2024
@grioda01
Copy link
Author

grioda01 commented Aug 19, 2024

Yes please, do you have an eta for its inclusion in releases?

@rogercoll
Copy link
Contributor

I would say the GeoIP processor already covers its basic functionality: add geographical attributes in the corresponding signal's context (resource attribute or record). In addition, a provider's configuration option is available with MaxMind provider as implementation.

In that sense, I think we could move the processor to alpha given that missing known features are related to extending the provider's functionalities. For example, allow MaxMind databases auto download.

There is just one missing issue I think it would be worth prioritizing before moving the processor to alpha: #34745 The issue implementation is a breaking change, many geographical attributes will be renamed.

@grioda01
Copy link
Author

As far as I am concerned, thinking that the geoip plugin was already operational, I already implemented the autodownload on my side of the maxmind db. So I don't need that extra feature of the geoip.
If we could at least have the geoip with it's current functionality, we could get somewhere, I don;t think that those extra feature are crucial, just a good to have. But without the geoip itself I am kind of block in implementing opentelemrtrycollector

@rogercoll
Copy link
Contributor

After discussing this topic with the geoip processor code owners, we agree on moving it to alpha. The processor is already functional with the current configuration, and feature functionalities should be additive.

@grioda01 Note that alpha component's can still perform breaking changes, for example, this issue #34745 will be a breaking change on the generated attributes.

Preparing PR to mark the component as alpha.

f7o pushed a commit to f7o/opentelemetry-collector-contrib that referenced this issue Sep 12, 2024
Moves its stability to alpha because the basic functionality is already
covered (local geoIP database + context to where apply the
transformations)

Closes open-telemetry#34737
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants