Collector is unable to find the file to export to even though the file exists.

[sai-sunder-s]

Component(s)

No response

What happened?

Description

I am experimenting with otel logging in my local machine. I want to export the logs to a file. I get file not found issue.

Steps to Reproduce

Expected Result

Wanted the collector to write the logs to a json file.

Actual Result

Got an error saying the file is not found. (Full Error in log output section below)

Collector version

v0.77.0

Environment information

Environment

OS: Debian GNU

OpenTelemetry Collector configuration

# otel-collector-config.yaml
receivers:
  otlp:
    protocols:
      grpc:
      http:

exporters:
  file:
    path: ~/data/playground/opentelemetry/otel-getting-started/logsotel.json
  logging:
    verbosity: detailed

processors:
  batch:

service:
  telemetry:
    logs:
      level: debug
   
  pipelines:
      logs:
          receivers: [otlp]
          processors: [batch]
          exporters: [file]

Log output

2023-05-18T05:06:50.529Z	info	service/telemetry.go:113	Setting up own telemetry...
2023-05-18T05:06:50.530Z	info	service/telemetry.go:136	Serving Prometheus metrics	{"address": ":8888", "level": "Basic"}
2023-05-18T05:06:50.530Z	debug	exporter@v0.77.0/exporter.go:284	Alpha component. May change in the future.	{"kind": "exporter", "data_type": "logs", "name": "file"}
Error: failed to build pipelines: failed to create "file" exporter for data type "logs": open ~/data/playground/opentelemetry/otel-getting-started/logsotel.json: no such file or directory
2023/05/18 05:06:50 collector server run finished with error: failed to build pipelines: failed to create "file" exporter for data type "logs": open ~/data/playground/opentelemetry/otel-getting-started/logsotel.json: no such file or directory

Additional context

No response

[atoulme]

The ~ symbol which is part of the path is used by bash to expand to the user home directory. The collector doesn't know how to interpret that path.

[github-actions]

Pinging code owners for exporter/file: @atingchen. See Adding Labels via Comments if you do not have permissions to add labels yourself.

[sai-sunder-s]

@atoulme I have tried replacing with the actual full path /usr/local/google/home/saisunder/data/playground/opentelemetry/otel-getting-started/logsotel.json and I get the same error.

[atoulme]

Does the folder /usr/local/google/home/saisunder/data/playground/opentelemetry/otel-getting-started exist?

[sai-sunder-s]

Yes the folder and the file exist.

On Thu, May 18, 2023 at 1:21 PM Antoine Toulme ***@***.***> wrote: Does the folder /usr/local/google/home/saisunder/data/playground/opentelemetry/otel-getting-started exist? — Reply to this email directly, view it on GitHub <#22065 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABCUPTIUW5HOKDGPODSK2O3XG2AEVANCNFSM6AAAAAAYF7PH4I> . You are receiving this because you authored the thread.Message ID: <open-telemetry/opentelemetry-collector-contrib/issues/22065/1553599677@ github.com>

-- with Regards Sai Sunder S

[sai-sunder-s]

If it is useful, here is the command I used to start the collector:

sudo docker run \
    -p 4317:4317 \
    -v $(pwd)/otel-collector-config.yaml:/etc/otelcol-contrib/config.yaml \
    otel/opentelemetry-collector-contrib:latest

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• exporter/file: @atingchen

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[crobert-1]

Hello @sai-sunder-s, when running the collector in a docker container it looks at paths internal to the container, not the host system. To be able to have access to your file you'd need to mount it to your container.

For example:
Update your config like this:

...
exporters:
  file:
    path: /etc/otelcol-contrib/logsotel.json
...

Run your container:

sudo docker run \
    -p 4317:4317 \
    -v $(pwd)/otel-collector-config.yaml:/etc/otelcol-contrib/config.yaml \
    -v /usr/local/google/home/saisunder/data/playground/opentelemetry/otel-getting-started/logsotel.json:/etc/otelcol-contrib/logsotel.json
    otel/opentelemetry-collector-contrib:latest

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• exporter/file: @atingchen

See Adding Labels via Comments if you do not have permissions to add labels yourself.