Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[exporter/loadbalancing] AWS Cloud Map to resolve collectors backends #27241

Closed
andretong opened this issue Sep 27, 2023 · 7 comments
Closed
Assignees
Labels

Comments

@andretong
Copy link
Contributor

andretong commented Sep 27, 2023

Component(s)

exporter/loadbalancing

Is your feature request related to a problem? Please describe.

Looking at the currently supported alternatives like k8s, it would be nice to add the AWS Service discovery system (CloudMap) to resolve the Collector's Backend.

This would allow users to use this exporter when there is the case of using ECS over EKS in an AWS infrastructure.

Describe the solution you'd like

  • Use the Go AWS SDK in order to use the Service Discovery service to fetch the IP addresses of the Collectors, and we could also get the port.
  • Attributes needed to make the requests, like namespace and serviceName will be defined as attributes in a new Resolver Config
  • Implement a new Resolver (aws_cloudmap_resolver.go for instance) based on the current interface
  • The implementation of the resolve method will apply the logic required to fetch and build the list by using the AWS SDK for Service Discovery.
  • This will resolve the list of backends periodically (i.e. every X amount of seconds and also configurable)

The configuration will look like this:

exporters:
  loadbalancing:
    protocol:
      otlp:
        timeout: 3s
    resolver:
      awsCloudMap:
        namespace: <aws-namespace>
        serviceName: <aws-otel-col-service-name>
        port: <port-to-override-the-default-> // https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/exporter/loadbalancingexporter/loadbalancer.go#L145
        interval: <interval-in-seconds>

Describe alternatives you've considered

No response

Additional context

No response

@andretong andretong added enhancement New feature or request needs triage New item requiring triage labels Sep 27, 2023
@github-actions
Copy link
Contributor

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@bryan-aguilar
Copy link
Contributor

@andretong I am going to assign this to you since I see you submitted an initial PR. Please feel free to unassign yourself if needed.

Copy link
Contributor

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions github-actions bot added the Stale label Dec 12, 2023
Copy link
Contributor

This issue has been closed as inactive because it has been stale for 120 days with no activity.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Feb 10, 2024
@jpkrohling jpkrohling reopened this Feb 13, 2024
@jpkrohling
Copy link
Member

Reopening, as there's a PR for this already.

mx-psi pushed a commit that referenced this issue Mar 13, 2024
…ter discovery (#27588)

**Description:** 
Implementation of
[27241](#27241)

Adding AWS Service discovery system (CloudMap) support to resolve the
Collector's Backend.
This would allow users to use this exporter when there is the case of
using ECS over EKS in an AWS infrastructure.

<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->

**Link to tracking Issue:** <Issue number if applicable>

**Testing:** <Describe what testing was performed and which tests were
added.>

Setting the following configuration with the proper values, we can start
the Load Balancer which will connect to the AWS CloudMap service and
start sending traces to the available collectors.

```yaml
exporters:
  loadbalancing:
    protocol:
      otlp:
        timeout: 3s
    resolver:
      awsCloudMap:
        namespace: <aws-namespace>
        serviceName: <aws-otel-col-service-name>
        interval: <interval-in-seconds>
```


**Documentation:** <Describe the documentation added.>
@atoulme
Copy link
Contributor

atoulme commented Mar 26, 2024

OK to close as complete? PR is merged.

@mx-psi
Copy link
Member

mx-psi commented Mar 27, 2024

I agree this seems fixed by #27588

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants