[aws_elb_otel] Add AWS ELB logs content pack for OpenTelemetry #15401

gpop63 · 2025-09-18T19:25:18Z

Overview

Adds documentation and dashboard for collecting AWS ELB access logs with the EDOT Cloud Forwarder.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.
I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

[ ]

How to test this PR locally

Related issues

Closes elastic/obs-integration-team/issues/583

Screenshots

…_pack

use esql queries in dashboard

ishleenk17 · 2025-09-22T04:18:15Z

@gpop63 : Can you also share the screenshot of Elastic ELB Logs dashboard.

packages/aws_elb_otel/docs/README.md

ishleenk17 · 2025-09-22T04:34:51Z

@gpop63 : Yet to review the dashboard. But shared other reviews.

The logo also, lets have one with ELB and OTEL image in the bottom. Refer to other OTEL Content Packs.

mykola-elastic · 2025-09-22T10:46:12Z

packages/aws_elb_otel/docs/README.md

+
+For the full documentation on how to set up the EDOT Cloud Forwarder, follow this link: [EDOT Cloud Forwarder for AWS](https://www.elastic.co/docs/reference/opentelemetry/edot-cloud-forwarder/aws).
+
+## Requirements


The doc team uses new namings for sections, like "What do I need to use this integration?" instead of "Requirements". There are more source

Though this is not an integration. Not sure if the new one is good here

ishleenk17 · 2025-09-22T11:51:30Z

packages/aws_elb_otel/docs/README.md

+* An Elastic **OTLP endpoint** and **API key** for authentication.
+
+## Setup
+


@gpop63 : We can mention like this for the config of ECF rather than mentioning the details in our CP.
Our CP aims to have dashboards. These config informations should be just FYI, high level

fix field names

ishleenk17 · 2025-09-22T13:52:36Z

NOTE: Once this PR is included in the ECOT Collector, we should change the dashboard filter to
scope.attributes.awslogs_encoding.format: elb_access_log

gpop63 · 2025-09-22T15:35:41Z

Can you also share the screenshot of Elastic ELB Logs dashboard.

@ishleenk17

This is Elastic ELB dashboard

OTEL dashboard

I have a widescreen thats why it seems more compact.

ishleenk17 · 2025-09-23T13:24:40Z

@gpop63 : Comments on the dashboard

Inbound and outbound traffic doens't showcase which one is inbound which is outbound. Also the y axis says response size. Is that correct ?
Y axis for status code should be Count.
Y axis for total requests should be Count.
Top IP addresses. Are we saying the IP addresses are in ocunt of 60K etc in the graph. Is there a better represeatation for this . Also, can we have maybe the top 10. Currently it looks cluttered.
Totle of dashboard doesn't have mention of OTEL in it.
In elastic dashboard I see different panels for ALB and ELB Inbound/outbound traffic. Are these different . Are the fields for these different ?

mykola-elastic · 2025-09-24T06:41:34Z

The first screenshot (integration page) in the PR description is low resolution

mykola-elastic · 2025-09-24T06:43:36Z

For the dashboard screenshot it would be good to have generate some data also in the 2xx and 5xx panels

gpop63 · 2025-09-26T09:58:08Z

@daniela-elastic updated the dashboard 👍

tommyers-elastic · 2025-09-30T10:05:11Z

small nit - can we not say "content pack" in the text? whilst this dashboard is getting shipped in a content pack, the dashboard itself is still just a dashboard.

elastic-sonarqube · 2025-09-30T10:59:53Z

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
97.7% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

packages/aws_elb_otel/manifest.yml

Co-authored-by: Mykola Kmet <mykola.kmet@elastic.co>

packages/aws_elb_otel/kibana/dashboard/aws_elb_otel-918bae2f-9268-46da-acac-7f4029ca0bfa.json

MichaelKatsoulis · 2025-10-20T07:09:19Z

packages/aws_elb_otel/kibana/dashboard/aws_elb_otel-918bae2f-9268-46da-acac-7f4029ca0bfa.json

+                        "meta": {
+                            "alias": null,
+                            "disabled": false,
+                            "field": "awslogs_encoding.format",


This field does not exist anymore

@gpop63 That is still not removed

MichaelKatsoulis · 2025-10-20T10:32:14Z

packages/aws_elb_otel/kibana/dashboard/aws_elb_otel-918bae2f-9268-46da-acac-7f4029ca0bfa.json

+                        "meta": {
+                            "alias": null,
+                            "disabled": false,
+                            "field": "awslogs_encoding.format",


@gpop63 That is still not removed

packages/aws_elb_otel/docs/README.md

MichaelKatsoulis

LGTM. Just a nit readme update!

Co-authored-by: Michalis Katsoulis <michaelkatsoulis88@gmail.com>

ishleenk17 · 2025-10-22T12:28:21Z

@gpop63 : I see that we have used the dataset filter at panel level and not global level.
Any reason for that ?

@gpop63 : I was checking in the dashboard, there is no mention of the word "OpenTelemetry" in the title, description of the dashboard. Anyone looking at just the dashboard will not get to know it is having OTEL data.

Do we have this present somewhere ?

gpop63 · 2025-10-22T14:03:48Z

@gpop63 : I see that we have used the dataset filter at panel level and not global level.
Any reason for that ?

Initially there was no data stream filter we could use globally, so I added filters at panel level for safety and consistency but we could remove them without any issue right now.

@gpop63 : I was checking in the dashboard, there is no mention of the word "OpenTelemetry" in the title, description of the dashboard. Anyone looking at just the dashboard will not get to know it is having OTEL data.

OTEL keyword is part of the dashboard title.

ishleenk17 · 2025-10-22T14:08:47Z

Initially there was no data stream filter we could use globally, so I added filters at panel level for safety and consistency but we could remove them without any issue right now.

Yes lets do that.

elasticmachine · 2025-10-22T20:41:05Z

💚 Build Succeeded

Buildkite Build
Commit: d0741aa

History

💚 Build #32938 succeeded b854f24
💚 Build #32937 succeeded 6c8c075
💚 Build #32882 succeeded 1314da5
💚 Build #32809 succeeded f601148
💚 Build #32109 succeeded 0632b31
💚 Build #32024 succeeded 741f76b

cc @gpop63

ishleenk17

Looks good!

elastic-vault-github-plugin-prod · 2025-10-23T08:48:50Z

Package aws_elb_otel - 0.1.0 containing this change is available at https://epr.elastic.co/package/aws_elb_otel/0.1.0/

…ic#15401) * add content pack * fix pr id * add codeowners entry * fix codeowners * rename integration to aws_elb_otel use esql queries in dashboard * fix codeowners entry * update logo * update dashboard title fix field names * update docs * add dashboard datastream filter * fix dashboard filter * address comments * use bytes for y axis * Update packages/aws_elb_otel/changelog.yml Co-authored-by: Mykola Kmet <mykola.kmet@elastic.co> * update dashboard * update dashboard * remove content pack from title * Update packages/aws_elb_otel/manifest.yml Co-authored-by: Mykola Kmet <mykola.kmet@elastic.co> * update data stream filter * fix datastream filters * fix datastream filter * Update packages/aws_elb_otel/docs/README.md Co-authored-by: Michalis Katsoulis <michaelkatsoulis88@gmail.com> * remove datastream filter at lens lvl --------- Co-authored-by: Mykola Kmet <mykola.kmet@elastic.co> Co-authored-by: Michalis Katsoulis <michaelkatsoulis88@gmail.com>

add content pack

7badf9f

gpop63 requested a review from a team as a code owner September 18, 2025 19:25

gpop63 self-assigned this Sep 18, 2025

gpop63 added New Integration Issue or pull request for creating a new integration package. dashboard Relates to a Kibana dashboard bug, enhancement, or modification. Team:Obs-InfraObs Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations] labels Sep 18, 2025

gpop63 requested a review from ishleenk17 September 18, 2025 19:26

gpop63 added 4 commits September 18, 2025 22:26

fix pr id

2cd67aa

add codeowners entry

4ffaba6

Merge remote-tracking branch 'upstream/main' into add_aws_elb_content…

74f6dac

…_pack

fix codeowners

09575f5

andrewkroh added the documentation Improvements or additions to documentation. Applied to PRs that modify *.md files. label Sep 18, 2025

rename integration to aws_elb_otel

97ebc00

use esql queries in dashboard

gpop63 changed the title ~~[elb_otel] Add AWS ELB logs content pack for OpenTelemetry~~ [aws_elb_otel] Add AWS ELB logs content pack for OpenTelemetry Sep 19, 2025

fix codeowners entry

2ee23f2

ishleenk17 reviewed Sep 22, 2025

View reviewed changes

packages/aws_elb_otel/docs/README.md Show resolved Hide resolved

packages/aws_elb_otel/docs/README.md Outdated Show resolved Hide resolved

packages/aws_elb_otel/docs/README.md Outdated Show resolved Hide resolved

packages/aws_elb_otel/docs/README.md Outdated Show resolved Hide resolved

mykola-elastic reviewed Sep 22, 2025

View reviewed changes

ishleenk17 reviewed Sep 22, 2025

View reviewed changes

gpop63 added 4 commits September 22, 2025 15:50

update logo

8568974

update dashboard title

f157310

fix field names

update docs

0e6892f

add dashboard datastream filter

f3d05bb

fix dashboard filter

3bce5d2

update dashboard

741f76b

daniela-elastic approved these changes Sep 30, 2025

View reviewed changes

remove content pack from title

0632b31

mykola-elastic reviewed Oct 16, 2025

View reviewed changes

packages/aws_elb_otel/manifest.yml Show resolved Hide resolved

gpop63 and others added 2 commits October 17, 2025 10:26

Update packages/aws_elb_otel/manifest.yml

2b39476

Co-authored-by: Mykola Kmet <mykola.kmet@elastic.co>

update data stream filter

f601148

MichaelKatsoulis reviewed Oct 20, 2025

View reviewed changes

packages/aws_elb_otel/kibana/dashboard/aws_elb_otel-918bae2f-9268-46da-acac-7f4029ca0bfa.json Outdated Show resolved Hide resolved

MichaelKatsoulis reviewed Oct 20, 2025

View reviewed changes

fix datastream filters

1314da5

gpop63 requested a review from MichaelKatsoulis October 20, 2025 07:44

MichaelKatsoulis requested changes Oct 20, 2025

View reviewed changes

fix datastream filter

6c8c075

MichaelKatsoulis self-requested a review October 21, 2025 08:41

MichaelKatsoulis reviewed Oct 21, 2025

View reviewed changes

packages/aws_elb_otel/docs/README.md Outdated Show resolved Hide resolved

MichaelKatsoulis approved these changes Oct 21, 2025

View reviewed changes

Update packages/aws_elb_otel/docs/README.md

b854f24

Co-authored-by: Michalis Katsoulis <michaelkatsoulis88@gmail.com>

remove datastream filter at lens lvl

d0741aa

ishleenk17 approved these changes Oct 23, 2025

View reviewed changes

lalit-satapathy approved these changes Oct 23, 2025

View reviewed changes

gpop63 merged commit 6bd3578 into elastic:main Oct 23, 2025
7 checks passed

andrewkroh added the Integration:aws_elb_otel AWS ELB OpenTelemetry Assets label Oct 23, 2025


		For the full documentation on how to set up the EDOT Cloud Forwarder, follow this link: [EDOT Cloud Forwarder for AWS](https://www.elastic.co/docs/reference/opentelemetry/edot-cloud-forwarder/aws).

		## Requirements

		* An Elastic OTLP endpoint and API key for authentication.

		## Setup

[aws_elb_otel] Add AWS ELB logs content pack for OpenTelemetry #15401

[aws_elb_otel] Add AWS ELB logs content pack for OpenTelemetry #15401

Uh oh!

Conversation

gpop63 commented Sep 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

Checklist

Author's Checklist

How to test this PR locally

Related issues

Screenshots

Uh oh!

ishleenk17 commented Sep 22, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ishleenk17 commented Sep 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mykola-elastic Sep 22, 2025

Choose a reason for hiding this comment

Uh oh!

mykola-elastic Sep 22, 2025

Choose a reason for hiding this comment

Uh oh!

ishleenk17 Sep 22, 2025

Choose a reason for hiding this comment

Uh oh!

ishleenk17 commented Sep 22, 2025

Uh oh!

gpop63 commented Sep 22, 2025

Uh oh!

ishleenk17 commented Sep 23, 2025

Uh oh!

mykola-elastic commented Sep 24, 2025

Uh oh!

mykola-elastic commented Sep 24, 2025

Uh oh!

gpop63 commented Sep 26, 2025

Uh oh!

tommyers-elastic commented Sep 30, 2025

Uh oh!

elastic-sonarqube bot commented Sep 30, 2025

Quality Gate passed

Uh oh!

Uh oh!

Uh oh!

MichaelKatsoulis Oct 20, 2025

Choose a reason for hiding this comment

Uh oh!

MichaelKatsoulis Oct 20, 2025

Choose a reason for hiding this comment

Uh oh!

MichaelKatsoulis Oct 20, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

MichaelKatsoulis left a comment

Choose a reason for hiding this comment

Uh oh!

ishleenk17 commented Oct 22, 2025

Uh oh!

gpop63 commented Oct 22, 2025

Uh oh!

ishleenk17 commented Oct 22, 2025

Uh oh!

elasticmachine commented Oct 22, 2025

💚 Build Succeeded

History

Uh oh!

ishleenk17 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

elastic-vault-github-plugin-prod bot commented Oct 23, 2025

Uh oh!

Reviewers

gpop63 commented Sep 18, 2025 •

edited

Loading

ishleenk17 commented Sep 22, 2025 •

edited

Loading