retreive instances via EC2 DescribeInstances

[trjstewart]

Is your feature request related to a problem? Please describe.
Currently the configuration options only allow for a single specific autoscaling group name to be specified per upstream with no use of wildcards. During our deployment process we blue/green autoscaling groups using Terraform which results in groups with names like nginx-regional-ingress-<unique_id>.

Describe the solution you'd like
Similar to the implementation in PR #6 I propose we replace the functionality of svcAutoscaling.DescribeAutoScalingGroups with svcEC2.DescribeInstances and allow the passing of an array of values to filter for on the aws:autoscaling:groupName tag (filtering supports wildcards). This will also satisfy #3.

region: self
api_endpoint: http://127.0.0.1/api
sync_interval_in_seconds: 1
cloud_provider: AWS
upstreams:
  - name: nginx-poc-api
    autoscaling_group: nginx-poc-*
    port: 3000
    kind: http
  - name: nginx-poc-api-v2
    autoscaling_group:
      - nginx-poc-api-one
      - nginx-poc-api-two
      - nginx-poc-api-*
    port: 3000
    kind: http

Describe alternatives you've considered
I am not aware of any other way to achieve the same functionality.

[pleshakov]

@trjstewart
The feature request makes sense. thanks!

However, perhaps we can extend it further to support more instance discovery options. Do you think it could be useful if we support any instance query, not just for autoscaling groups? Consider the following example:

region: us-west-2
api_endpoint: http://127.0.0.1:8080/api
sync_interval_in_seconds: 5
cloud_provider: AWS
upstreams:
- name: backend-one
  autoscaling_group: backend-one-group
  port: 80
  kind: http
- name: backend-two
  instance_query:
    tag:aws:autoscaling:groupName:
    - group-1
    - group-2
  port: 80
  kind: http
- name: backend-three
  instance_query:
    tag:service:
    - myservice
    tag:version:
    - v1
  port: 80
  kind: http

In this case, instance_query field contains a list of filters, as defined in https://docs.aws.amazon.com/sdk-for-go/api/service/ec2/#DescribeInstancesOutput

[trjstewart]

I did consider making the feature more generalized to allow for far more granular filtering of returned instances. At the time I opted away from that and stuck with specifically filtering on only the aws:autoscaling:groupName tag mainly due to the name and purpose of the repository.

If you'd be happy to extend the overall functionality of the repository to be more generalized I'd be happy to put through a PR for it.

[pleshakov]

@trjstewart I think it makes sense to extend the functionality so that the asg-sync agent can discover instances in general, not only instances that belong to auto scaling groups. Although, as you mentioned, this extension will go against the current name and purpose of the repository.

That'd be great if you create a PR for it. We will update the description and the documentation of the repo accordingly.

[trjstewart]

@pleshakov I have submitted a PR (#29) to satisfy my initial proposal as I need this feature to continue some other work for our NGINX+ POC. I believe this is a good starting point while we further discuss the overall implementation of adding flexible instance filtering via tags as that will likely require considerable changes to the config schema as well as the CloudProvider interface.

This initial PR will satisfy the requirements I need for work but I would still like to build the feature out completely in my own time. I'll put together a draft PR for the config and interface changes over the next few days.

[pleshakov]

@trjstewart

This initial PR will satisfy the requirements I need for work but I would still like to build the feature out completely in my own time. I'll put together a draft PR for the config and interface changes over the next few days.

#29 is a good extension that preserves backward compatibility.

We'd be happy to review the draft. Please take your time.

[Rulox]

Implemented in #29 thanks.