Skip to content

Conversation

@mergify
Copy link

@mergify mergify bot commented Sep 23, 2025

Proposed Changes

AWS EC2 API returns networkInterfaceSet and privateIpAddressesSet in arbitrary order, causing non-deterministic hostname resolution during peer discovery and inconsistent cluster formation when setting certain cluster_formation.aws.hostname_path configurations such as:

cluster_formation.aws.hostname_path = networkInterfaceSet,2,privateIpAddressesSet,1,privateIpAddress

See https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeInstances.html

"The order of the elements in the response, including those within nested structures, might vary. Applications should not assume the elements appear in a particular order."

Changes:

  • Sort network interfaces by deviceIndex (0 first for primary ENI)
  • Sort private IP addresses by primary flag (primary=true first)
  • Add debug logging to show hostname path selection and sorting results (removed in second revision)
  • Add unit tests for sorting behavior

The sorting ensures deviceIndex=0 and primary=true IPs are consistently selected as intended, making peer discovery deterministic across deployments. Note that since EC2 does not provide an index for privateIpAddressesSet (like deviceIndex for networkInterfaceSet), this determinism for privateIpAddressesSet only applies if there are 1 or 2 members in privateIpAddressesSet via the primary=true or primary=false. If there was a 3rd member in privateIpAddressesSet, the two primary=false IPs orders are non-deterministic.

Types of Changes

What types of changes does your code introduce to this project?
Put an x in the boxes that apply

  • Bug fix (non-breaking change which fixes issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause an observable behavior change in existing systems)
  • Documentation improvements (corrections, new content, etc)
  • Cosmetic change (whitespace, formatting, etc)
  • Build system and/or CI

Checklist

Put an x in the boxes that apply.
You can also fill these out after creating the PR.
This is simply a reminder of what we are going to look for before merging your code.

  • Mandatory: I (or my employer/client) have have signed the CA (see https://github.com/rabbitmq/cla)
  • I have read the CONTRIBUTING.md document
  • I have added tests that prove my fix is effective or that my feature works
  • All tests pass locally with my changes
  • If relevant, I have added necessary documentation to https://github.com/rabbitmq/rabbitmq-website
  • If relevant, I have added this change to the first version(s) in release-notes that I expect to introduce it

Further Comments

N/A


This is an automatic backport of pull request #14557 done by [Mergify](https://mergify.com).

* AWS peer discovery: ensure consistent hostname path ordering

AWS EC2 API returns networkInterfaceSet and privateIpAddressesSet in
arbitrary order, causing non-deterministic hostname resolution during
peer discovery. This leads to inconsistent cluster formation.

Changes:
- Sort network interfaces by deviceIndex (0 first for primary ENI)
- Sort private IP addresses by primary flag (primary=true first)
- Add debug logging to show hostname path selection and sorting results
- Add comprehensive unit tests for sorting behavior

The sorting ensures deviceIndex=0 and primary=true IPs are consistently
selected first, making peer discovery deterministic across deployments.

* AWS peer discovery: ensure consistent hostname path ordering (address feedback on debug logs and sorting helper functions)

(cherry picked from commit 4a32470)
@michaelklishin michaelklishin added this to the 4.2.0 milestone Sep 24, 2025
@michaelklishin michaelklishin merged commit df5d6e2 into v4.2.x Sep 24, 2025
566 of 568 checks passed
@michaelklishin michaelklishin deleted the mergify/bp/v4.2.x/pr-14557 branch September 24, 2025 03:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants