Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

config is evaluated twice. having to double up on $ #628

Closed
rockb1017 opened this issue Aug 16, 2021 · 6 comments
Closed

config is evaluated twice. having to double up on $ #628

rockb1017 opened this issue Aug 16, 2021 · 6 comments
Assignees
Labels
bug Something isn't working

Comments

@rockb1017
Copy link
Contributor

when using filelog receiver, $ is used for accessing data from entry.
https://github.com/open-telemetry/opentelemetry-log-collection/blob/main/docs/types/field.md
and i have an example config here.

  filelog:
    encoding: nop
    exclude:
    - /var/log/pods/default_rock-sck-otel*_*/otelcollector/*.log
    - /var/log/pods/kube-system_*/*/*.log
    fingerprint_size: 1kb
    include:
    - /var/log/pods/*/*/*.log
    include_file_name: false
    include_file_path: true
    max_concurrent_files: 1024
    max_log_size: 1MiB
    operators:
    - id: get-format
      routes:
      - expr: $$body matches "^\\{"
        output: parser-docker
      - expr: $$body matches "^[^ Z]+ "
        output: parser-crio
      - expr: $$body matches "^[^ Z]+Z"
        output: parser-containerd
      type: router
    - id: parser-crio
      regex: ^(?P<time>[^ Z]+) (?P<stream>stdout|stderr) (?P<logtag>[^ ]*) (?P<log>.*)$
      timestamp:
        layout: "2006-01-02T15:04:05.000000000-07:00"
        layout_type: gotime
        parse_from: time
      type: regex_parser
    - combine_field: log
      id: crio-recombine
      is_last_entry: ($.logtag) == 'F'
      type: recombine
    - id: check for empty log
      ops:
      - add:
          field: log
          if: EXPR($.log) != nil
          value: nil
      output: filename
      type: restructure
    - id: parser-containerd
      regex: ^(?P<time>[^ ^Z]+Z) (?P<stream>stdout|stderr) (?P<logtag>[^ ]*) (?P<log>.*)$
      timestamp:
        layout: '%Y-%m-%dT%H:%M:%S.%LZ'
        parse_from: time
      type: regex_parser
    - combine_field: log
      id: containerd-recombine
      is_last_entry: ($.logtag) == 'F'
      output: filename
      type: recombine
    - id: parser-docker
      timestamp:
        layout: '%Y-%m-%dT%H:%M:%S.%LZ'
        parse_from: time
      type: json_parser
    - id: filename
      resource:
        service.name: EXPR($$attributes["file.path"])
      type: metadata
    - id: extract_metadata_from_filepath
      parse_from: $$attributes["file.path"]
      regex: ^\/var\/log\/pods\/(?P<namespace>[^_]+)_(?P<pod_name>[^_]+)_(?P<uid>[^\/]+)\/(?P<container_name>[^\._]+)\/(?P<run_id>\d+)\.log$
      type: regex_parser
    - resource:
        com.splunk.sourcetype: EXPR("kube:container:"+$.container_name)
        container_name: EXPR($.container_name)
        k8s.namespace.name: EXPR($.namespace)
        k8s.pod.name: EXPR($.pod_name)
        k8s.pod.uid: EXPR($.uid)
        run_id: EXPR($.run_id)
        stream: EXPR($.stream)
      type: metadata
    - id: clean-up-log-record
      ops:
      - move:
          from: log
          to: $
      type: restructure
    poll_interval: 200ms
    resource:
      test-scenaro: 8198:200
    start_at: beginning

but the $$body is parsed and changed to env var value of body. So we can't use image from this repo for unified helm chart. I think some changes are required in the cmd/otelcol/main.go

@rockb1017
Copy link
Contributor Author

@dmitryax
Copy link
Contributor

It seems like we have a bug somewhere in https://github.com/signalfx/splunk-otel-collector/tree/main/internal/configprovider . core/contrib works well

@dmitryax dmitryax added the bug Something isn't working label Aug 17, 2021
@dmitryax
Copy link
Contributor

dmitryax commented Aug 17, 2021

Did some initial investigation.

Looks like config is evaluated twice in our distro.

Any $$key is evaluated to $key by configprovider.Manager.Resolve first (and shown correctly in http://localhost:55554/debug/configz/effective), but then components get config with an empty value or any other value set for $key env var.

@rockb1017
Copy link
Contributor Author

Thanks @dmitryax from what you shared, I tried something and it worked. I doubled up $. and everything worked as it should.

@rockb1017
Copy link
Contributor Author

should we fix it or just use $$$$ in the helm chart?

@dmitryax
Copy link
Contributor

should we fix it or just use $$$$ in the helm chart?

we can do that as an ugly short term workaround to unblock moving to our image. I'll try to fix this bug in the meantime

@dmitryax dmitryax self-assigned this Aug 18, 2021
@rockb1017 rockb1017 changed the title cannot use config for filelog receiver expressions config is evaluated twice. having to double up on $ Nov 23, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants