Any end-to-end tutorial for OpAMP in OTel Collector?

[brightzheng100]

Component(s)

extension/opamp

Is your feature request related to a problem? Please describe.

Even OpAMP in Otel is still in early days but it is really cool so I wanted to have a try by picking up some pieces of the info scattered into the Otel projects.

What I've tried is something like this:

1. Spin up the OpAMP server:

git clone https://github.com/open-telemetry/opamp-go.git
make build-example-server
cd internal/examples/server
./bin/server

Then I can access the UI at: http://127.0.0.1:4321/

2. Customize the Otel Collector:

Open another console.

cat <<EOF > builder-config.yaml
dist:
  name: otelcol-dev
  description: Basic OTel Collector distribution for Developers
  output_path: ./otelcol-dev
  otelcol_version: 0.88.0

exporters:
  - gomod:
      # debug
      go.opentelemetry.io/collector/exporter/debugexporter v0.88.0
  - gomod:
      # otlp
      go.opentelemetry.io/collector/exporter/otlpexporter v0.88.0

receivers:
  - gomod:
      # otlp
      go.opentelemetry.io/collector/receiver/otlpreceiver v0.88.0

processors:
  - gomod:
      # batch
      go.opentelemetry.io/collector/processor/batchprocessor v0.88.0

extensions:
  - gomod:
      # opamp
      github.com/open-telemetry/opentelemetry-collector-contrib/extension/opampextension 22d86de838858c25d1d644b84221eb37198d2c86
EOF

./ocb --config builder-config.yaml

3. Configure the Collector and start it up:

cat <<EOF > config.yaml
receivers:
  otlp:
    protocols:
      grpc:

exporters:
  debug:
    verbosity: detailed

extensions:
  opamp:
    server:
      ws:
        endpoint: wss://127.0.0.1:4320/v1/opamp

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [debug]
  telemetry:
    logs:
      level: debug
EOF

./otelcol-dev/otelcol-dev --config config.yaml

But there sounds no effect: everything looks okay but nothing happens, in both UI and Collector.

So, is there an end-to-end tutorial that I can read and follow through?

Describe the solution you'd like

An end-to-end tutorial about how OpAMP in Otel.

Describe alternatives you've considered

No response

Additional context

No response

[github-actions]

Pinging code owners:

• extension/opamp: @portertech @evan-bradley @tigrannajaryan

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[brightzheng100]

The opamp-supervisor is slightly more interesting at this moment.

1. Build opamp-supervisor

git clone https://github.com/open-telemetry/opentelemetry-collector-contrib.git
cd opentelemetry-collector-contrib/cmd/opampsupervisor
go build -o ~/workspaces/OSS/OpenTelemetry/playground/opamp-supervisor

2. Configure supervisor-config.yaml

cd ~/workspaces/OSS/OpenTelemetry/playground

cat <<EOF > supervisor-config.yaml
server:
  endpoint: wss://127.0.0.1:4320/v1/opamp
  tls:
    insecure_skip_verify: true 

capabilities:
  reports_effective_config: true
  reports_own_metrics: true
  reports_health: true
  accepts_remote_config: true
  reports_remote_config: true

agent:
  executable: ./otelcol-contrib
  config_file: ./config.yaml
EOF

3. Start up opamp-supervisor

./opamp-supervisor --config supervisor-config.yaml

4. Check out the UI

I found that the effective configuration is not what I injected so I roughly checked out the code and found that the feature of accepting OTel Collector config file might not be ready yet.

[tigrannajaryan]

There is no tutorial that I am aware of. You are doing the right things, the Collector is just not yet ready :-)
Both opampextension and Supervisor are work-in-progress.

But there sounds no effect: everything looks okay but nothing happens, in both UI and Collector.

You should also see the connected Agent in the Server's UI when using just the opamp extension without the Supervisor. If you don't then this is a bug, please file it as a separate issue. Note that you should expect anything to happen in the Collector. The opamp extension is purely readonly at the moment.

I found that the effective configuration is not what I injected so I roughly checked out the code and found that the feature of accepting OTel Collector config file might not be ready yet.

This should work as far as I know. You may want to open a separate issue and describe what exactly is wrong.

[crobert-1]

It looks like there are a few different things going on here, but we'll keep this issue open as a request for an end-to-end example of how to use the opamp extension.

[brightzheng100]

You should also see the connected Agent in the Server's UI when using just the opamp extension without the Supervisor.

You're right. I should activate such an extension by adding it as part of the service section, like this:

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [debug]
  telemetry:
    logs:
      level: debug
  extensions: [opamp]   # <-- I missed this part

But the effective configuration isn't what I configured in the Collector, so I will raise it in another issue as a bug.

What I configured now is:

receivers:
  otlp:
    protocols:
      grpc:

exporters:
  debug:
    verbosity: detailed

extensions:
  opamp:
    server:
      ws:
        endpoint: wss://127.0.0.1:4320/v1/opamp
        tls:
          insecure_skip_verify: true
    instance_uid: 01ARZ3NDEKTSV4RRFFQ69G5FAV

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [debug]
  telemetry:
    logs:
      level: debug
  extensions: [opamp]

And what I saw in the Effective Configuration in http://127.0.0.1:4321/agent?instanceid=01ARZ3NDEKTSV4RRFFQ69G5FAV is:

exporters:
  otlp:
    endpoint: localhost:1111
receivers:
  otlp:
    protocols:
      grpc: {}
      http: {}
service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: []
      exporters: [otlp]

[brightzheng100]

Raised a new issue for the "effective configuration": #29117

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• extension/opamp: @portertech @evan-bradley @tigrannajaryan

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

This issue has been closed as inactive because it has been stale for 120 days with no activity.

[MSA0208]

Hi have tried the same steps given by you @brightzheng100 ,
but getting the below error, i tried starting the server from /root/OTEL98/opamp-go-main/internal/examples/server and it started and when i tried the extension from my collector getting below error
extensions:
opamp:
server:
ws:
endpoint: wss://127.0.0.1:4321/v1/opamp
tls:
insecure_skip_verify: false # Set to true if you're using self-signed certificates
ca_file: "/root/certs/sac-hvm03662.swinfra.net.pem"

error opampextension@v0.98.0/logger.go:26 Connection failed (tls: first record does not look like a TLS handshake), will retry. {"kind": "extension", "name": "opamp", "client": "ws"}

2. The Other way i tried was , started the opampsupervisor from /root/OTEL98/opentelemetry-collector-contrib-main/cmd/opampsupervisor using the command ./opampsupervisor --config supervisor.yaml and my yaml is below

OpAMP backend server settings.

server:
endpoint: wss://127.0.0.1:4320/v1/opamp
tls:
insecure_skip_verify: true

capabilities:
accepts_remote_config: true # false if unspecified

reports_effective_config: true # true if unspecified

accepts_packages: true # false if unspecified

reports_own_metrics: # true if unspecified

reports_own_logs: # true if unspecified

accepts_other_connection_settings: true # false if unspecified

accepts_restart_command: true # true if unspecified

reports_health: true # true if unspecified

storage:
agent:
executable: /root/OTEL98/opentelemetry-collector-contrib-main/cmd/otelcontribcol/OutputBinaries/NGxConnector

args: --config /root/OTEL98/opentelemetry-collector-contrib-main/cmd/otelcontribcol/config.yaml

env:

run_as: root
config_file: /root/OTEL98/opentelemetry-collector-contrib-main/cmd/otelcontribcol/config.yaml
access_dirs:
read:
allow: [/var/log]
deny: [/var/log/secret_logs]
write:
allow: [/var/otelcol]
am getting the error below,
[root@sac-hvm03662 opampsupervisor]# ./opampsupervisor --config supervisor.yaml
2024-05-07T10:40:22.536-0700 DEBUG commander/commander.go:74 Starting agent {"agent": "/root/OTEL98/opentelemetry-collector-contrib-main/cmd/otelcontribcol/OutputBinaries/NGxConnector"}
2024-05-07T10:40:22.537-0700 DEBUG commander/commander.go:93 Agent process started {"pid": 63200}
2024-05-07T10:40:25.540-0700 ERROR opampsupervisor/main.go:31 Failed to initialize supervisor {"error": "could not get bootstrap info from the Collector: collector's OpAMP client never connected to the Supervisor"}
main.main
/root/OTEL98/opentelemetry-collector-contrib-main/cmd/opampsupervisor/main.go:31
runtime.main
/usr/local/go/src/runtime/proc.go:271

please help me out to solve these issues

[priyeshsingh550]

Hi I am also facing the same issue @MSA0208 have you got any solution for it??

Please help me also to solve this issue.

[MSA0208]

hi @priyeshsingh550 ,

No, Even am continuing my R&D on it, but got few suggestions from #34377 tried it, still the same issues.
#34377

Let me know if something from the above works for you.