Skip to content

fix(ci): use localhost endpoints in LocalStack demo to avoid DNS hangs - #2598

Merged
Andriy Knysh (aknysh) merged 1 commit into
mainfrom
osterman/fix-localstack-dns-endpoints
Jun 10, 2026
Merged

Andriy Knysh (aknysh) merged 1 commit into
mainfrom
osterman/fix-localstack-dns-endpoints

Conversation

@osterman

@osterman Erik Osterman (Cloud Posse) (osterman) commented Jun 10, 2026 •

Copy link
Copy Markdown
Member

what

  • Switch the demo-localstack example's AWS provider endpoints from https://localhost.localstack.cloud:4566 to http://localhost:4566
  • Enable s3_use_path_style: true so S3 operations don't depend on wildcard *.localhost.localstack.cloud DNS/TLS
  • Add skip_requesting_account_id: true so provider configure doesn't block on identity lookups
  • Update the example README to document the settings and the expected "AWS account ID not found for provider" warning

why

The [localstack] demo-localstack CI job started hanging until the 20-minute timeout across all branches beginning 2026-06-10 ~02:18 UTC, with no corresponding code change (identical commits both passed and failed; the LocalStack image is pinned at 1.4.0 with the same build hash in green and red runs).

Root cause: localhost.localstack.cloud is a public DNS record hosted by LocalStack that resolves to 127.0.0.1. After LocalStack EOL'd Community Edition (repo archived March 2026), their DNS zone was restructured on 2026-06-08 18:53 UTC (SOA serial; the record is now a freshly delegated Route53 subzone). GitHub's Azure runners began intermittently failing to resolve the name ~31h later as resolver caches expired. The Terraform AWS provider treats DNS failure as retryable and backs off past the job timeout — hanging silently before its first API call.

Evidence:

  • Every failed run hangs at the identical point: after terraform workspace new dev-demo, before the provider's first API call. The LocalStack container log shows exactly one completed request (sts.GetSessionToken => 200 — issued by atmos auth, which uses http://localhost:4566 and always succeeds). The provider's first call via localhost.localstack.cloud never arrives.
  • A/B proof: on osterman/fix-post-merge-sha, a run with the old endpoints timed out at 13:43 UTC; the identical change in this PR passed at 13:53 UTC (run 27281214186).

Extending the timeout would not help — the provider's retry backoff exceeds any reasonable limit when DNS is failing.

references

  • Failing runs (examples): 27248701025, 27283701712, 27282384395, 27300390150 — all cancelled at the 20-min [localstack] job timeout
  • Green run with this exact change: 27281214186
  • LocalStack Community EOL: https://blog.localstack.cloud/the-road-ahead-for-localstack/ (the DNS zone change itself is unannounced)

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Documentation
    • Updated LocalStack example documentation and configuration for improved CI compatibility, clarifying expected Terraform behavior and configuration settings.

The demo-localstack CI job began timing out at the 20-minute limit across
all branches on 2026-06-10. Root cause: the provider endpoints used
https://localhost.localstack.cloud:4566, a public DNS record hosted by
LocalStack. After LocalStack EOL'd Community Edition, their DNS zone was
restructured on 2026-06-08 (localhost.localstack.cloud re-delegated to a
new Route53 subzone), and GitHub's Azure runners now intermittently fail
to resolve it. The AWS provider treats DNS failure as retryable and backs
off past the job timeout, hanging silently before its first API call.

Switch all endpoints to http://localhost:4566 (the port the GitHub Actions
service container actually exposes), enable path-style S3 so bucket
operations don't depend on wildcard *.localhost.localstack.cloud DNS/TLS,
and skip the account-id lookup during provider configure.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@atmos-pro

atmos-pro Bot commented Jun 10, 2026 •

Copy link
Copy Markdown
Contributor

Tip

Atmos Pro  

No affected stacks workflow was detected for this pull request.
If this is expected, no action is needed.
Learn More. Ask AI.

@osterman Erik Osterman (Cloud Posse) (osterman) added the no-release Do not create a new release (wait for additional code changes) label Jun 10, 2026
@github-actions github-actions Bot added the size/s Small size PR label Jun 10, 2026
@github-actions

Copy link
Copy Markdown

Dependency Review

✅ No vulnerabilities or license issues found.

Scanned Files

None

@coderabbitai

coderabbitai Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 96c824ff-ab66-42a9-b425-d05e45b5156b

📥 Commits

Reviewing files that changed from the base of the PR and between aa223e0 and 0db0a7a.

📒 Files selected for processing (2)
  • examples/demo-localstack/README.md
  • examples/demo-localstack/stacks/mixins/localstack.yaml

📝 Walkthrough

Walkthrough

The LocalStack demo example is updated to use HTTP endpoints with path-style S3 instead of HTTPS. The Terraform provider configuration switches the shared LocalStack URL from TLS to plain HTTP and consolidates S3 endpoint wiring. The README is updated to document this choice and clarify that the resulting Terraform warning is expected.

Changes

LocalStack Example Configuration Update

Layer / File(s) Summary
LocalStack path-style S3 and HTTP endpoint configuration
examples/demo-localstack/stacks/mixins/localstack.yaml, examples/demo-localstack/README.md
The AWS provider switches the LocalStack endpoint URL alias from HTTPS to HTTP, enables S3 path-style access (s3_use_path_style: true), and consolidates S3 endpoint wiring to use the shared URL alias. The README is updated to document path-style S3 against http://localhost:4566, note skip_requesting_account_id usage to avoid provider hangs, and clarify that the Terraform warning is expected under this configuration.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Suggested labels

no-release

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title directly and specifically describes the main fix: switching to localhost endpoints in the LocalStack demo to resolve DNS/timeout issues in CI.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch osterman/fix-localstack-dns-endpoints

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov

codecov Bot commented Jun 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 79.03%. Comparing base (aa223e0) to head (0db0a7a).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #2598      +/-   ##
==========================================
+ Coverage   79.01%   79.03%   +0.02%     
==========================================
  Files        1210     1210              
  Lines      117219   117219              
==========================================
+ Hits        92620    92647      +27     
+ Misses      19521    19492      -29     
- Partials     5078     5080       +2     
Flag Coverage Δ
unittests 79.03% <ø> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 6 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@aknysh
Andriy Knysh (aknysh) merged commit 7064397 into main Jun 10, 2026
67 checks passed
@aknysh
Andriy Knysh (aknysh) deleted the osterman/fix-localstack-dns-endpoints branch June 10, 2026 22:04
@atmos-pro

atmos-pro Bot commented Jun 10, 2026 •

Copy link
Copy Markdown
Contributor

Tip

Atmos Pro  

No affected stacks workflow was detected for this pull request.
If this is expected, no action is needed.
Learn More. Ask AI.

Andriy Knysh (aknysh) added a commit that referenced this pull request Jun 11, 2026
Resolve modify/delete conflicts on the renamed example:

main #2598 ("use localhost endpoints in LocalStack demo to avoid DNS hangs")
modified examples/demo-localstack/README.md and stacks/mixins/localstack.yaml,
which this branch deleted as part of renaming demo-localstack -> demo-floci.

Resolved by keeping the deletions: the same fixes (s3_use_path_style: true,
skip_requesting_account_id: true, http://localhost:4566 path-style endpoints,
and the expected-warning note) already exist in the renamed files
examples/demo-floci/stacks/mixins/floci.yaml and examples/demo-floci/README.md,
so nothing from #2598 is lost.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Igor Rodionov (goruha) added a commit that referenced this pull request Jun 11, 2026
…atmos into 1198-pact-consumer-contracts

* '1198-pact-consumer-contracts' of github.com:cloudposse/atmos:
  fix(ansible): forward `-- <args>` passthrough to ansible-playbook (#2594)
  fix(ci): use localhost endpoints in LocalStack demo to avoid DNS hangs (#2598)
  DAG follow-up: aggregate CI output for concurrent Terraform runs (#2577)
  fix(list): prevent nil pointer panic when spinner exits early (#2591)
  feat: describe affected evaluates all provisioned component sections (#2573)
@github-actions

Copy link
Copy Markdown

These changes were released in v1.221.1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-release Do not create a new release (wait for additional code changes) size/s Small size PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants