Adding in AWS tests using providertest framework

[rshade]

This pull request updates several GitHub Actions workflows to use versioned action tags (like @v4 or @v1) instead of full commit SHAs, and introduces a new workflow for AWS EKS testing. The main goals are to simplify maintenance, keep actions up-to-date, and add automated EKS test coverage.

Workflow maintenance and modernization:

• Updated all referenced GitHub Actions in workflow files (such as auto-approve.yml, main.yml, pr-test.yml, and test-with-minio.yml) to use versioned tags (e.g., @v4, @v1, etc.) instead of hardcoded commit SHAs, making future updates easier and reducing risk of using outdated actions. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11]

• Replaced deprecated or renamed actions with their recommended alternatives, such as switching from pascalgn/merge-action to pascalgn/automerge-action, and from actions/create-release and actions/upload-release-asset to shogo82148/actions-create-release and shogo82148/actions-upload-release-asset. [1] [2] [3]

New AWS EKS test workflow:

• Added a new workflow file .github/workflows/test-aws-eks.yml to automate AWS EKS tests on pull requests and workflow dispatch, including environment setup, AWS credentials, Pulumi CLI installation, test execution, and artifact upload. This workflow triggers when relevant files change or the test:aws-eks label is present.

These changes collectively improve workflow reliability, security, and test coverage for AWS EKS.

[rshade]

You can use commands like esc run team-ce/default/aws make test-aws-ecs-go to test self-hosted stacks.

[claude]

Claude finished @rshade's task —— View job

---

Pull Request Review: AWS EKS Testing Framework

I've completed a comprehensive review of this PR that introduces AWS EKS testing infrastructure and GitHub Actions workflow improvements. Here's my detailed analysis:

Review Checklist:

• Analyze the new test framework architecture in tests/ directory
• Review AWS EKS configuration changes for best practices
• Evaluate GitHub Actions workflow improvements and security
• Check Pulumi Infrastructure-as-Code patterns and conventions
• Assess security practices and credential management
• Validate deployment stage dependencies and stack references
• Review code quality and documentation updates
• Check Makefile and build automation

🟢 Strengths and Best Practices

1. Excellent Test Framework Architecture

• Robust ESC Token Management: The RefreshEscToken() and ExecuteWithRetry() utilities in tests/utils.go:114-220 are excellent for handling AWS credential expiration in long-running tests
• LIFO Cleanup Pattern: The CleanupStacksWithRetry() function (lines 235-270) properly destroys resources in reverse order, preventing dependency issues
• Environment Isolation: The TestEnvironment pattern provides proper test isolation with unique stack prefixes
• Comprehensive Service Validation: The service_validation.go module provides thorough end-to-end testing including user creation and organization management

2. Production-Ready Configuration Management

• Configurable Resource Protection: The protectResources configuration added across all EKS stages allows tests to disable protection while keeping production deployments protected by default
• Proper Stack References: Tests properly use opttest.UseAmbientBackend() for stack reference compatibility
• AWS Default Tags: Consistent tagging strategy with Purpose, AutoDelete, CreatedBy tags for resource management

3. Security Improvements

• GitHub Actions Version Tags: Excellent update from commit SHAs to version tags (e.g., @v4, @v1.2.0) for better security and maintainability
• Environment-based Secrets: Proper use of GitHub Actions environments for credential management
• Credential Validation: Environment variable validation functions prevent tests from running with missing credentials

🟡 Areas for Improvement

1. GitHub Actions Workflow Security

# In .github/workflows/main.yml:15
- uses: actions/checkout@ff7abcd0c3c05ccf6adc123a8cd1fd4fb30fb493 # v4

Issue: This still uses a commit SHA instead of the updated version tag pattern. Should be:

- uses: actions/checkout@v4

2. AWS Region Configuration

// In tests/aws_eks_ts_test.go:33
config := map[string]string{
    "aws:region": "us-east-1",
}

Recommendation: Consider making the AWS region configurable via environment variable to support multi-region testing and align with the ca-central-1 requirement mentioned in CLAUDE.md.

3. Test Timeout and Resource Limits

# In .github/workflows/test-aws-eks.yml:27
timeout-minutes: 120

Consideration: 120 minutes may be insufficient for the full 8-stage EKS deployment. Based on CLAUDE.md, EKS tests typically take ~2 hours. Consider increasing to 150 minutes to account for potential delays.

🔴 Critical Security Considerations

1. ESC Authentication Pattern

The CLAUDE.md mentions that AWS commands must use ESC authentication:

# Correct
esc run team-ce/default/aws go test -v -timeout=2h -tags aws ./...

However, the GitHub Actions workflow runs tests directly without ESC wrapper. Ensure the CI environment has proper AWS credentials configured.

2. Resource Cleanup Verification

The test framework has excellent cleanup utilities, but consider adding post-test verification to ensure no resources are left behind, especially important for the shared ca-central-1 account mentioned in CLAUDE.md.

📋 Deployment Pattern Validation

✅ Excellent Stage Dependencies

The test properly follows the 8-stage EKS deployment pattern:

1. 01-iam → 2. 02-networking → 3. 05-eks-cluster → 4. 10-cluster-svcs → 5. 15-state-policies-mgmt → 6. 20-database → 7. 25-insights → 8. 30-esc → 9. 90-pulumi-service

✅ Stack Reference Compatibility

Tests properly use shared backend for cross-stack references with UseAmbientBackend().

🛠️ Code Quality Assessment

Strengths:

• Go code follows proper error handling patterns with detailed logging
• Comprehensive test coverage with service validation
• Proper use of testify assertions
• Good separation of concerns between utilities and test logic

Minor Issues:

• Some unused functions have proper //nolint:unused comments
• File paths are properly handled with filepath.Join()
• Context handling is appropriate for Pulumi operations

📊 Documentation and Maintainability

✅ Excellent Documentation

• tests/README.md provides comprehensive usage instructions
• Clear build tag documentation for conditional compilation
• Proper Makefile targets for different test scenarios

Suggestion:

Consider adding a troubleshooting section to the README for common test failures (token expiration, resource quota limits, etc.).

🎯 Final Recommendations

1. Fix remaining commit SHA in main.yml workflow
2. Consider making AWS region configurable for multi-region testing
3. Increase GitHub Actions timeout to 150 minutes for safety
4. Add post-test resource verification to prevent leakage
5. Consider adding integration with the Makefile targets mentioned in CLAUDE.md

✅ Overall Assessment

This is an excellent pull request that significantly improves the testing infrastructure for Pulumi Self-Hosted installers. The token management utilities, cleanup patterns, and service validation framework are particularly well-designed. The GitHub Actions improvements enhance security and maintainability. With the minor fixes suggested above, this PR provides a robust foundation for continuous testing of the EKS deployment.

Recommendation: Approve with minor fixes

---

[claude]

Claude finished @rshade's task —— View job

---

Pull Request Review: AWS EKS Testing Framework

I've completed a comprehensive review of this PR that introduces AWS EKS testing infrastructure and GitHub Actions workflow improvements. Here's my detailed analysis:

Review Checklist:

• Read repository guidelines and understand codebase structure
• Analyze the new test framework architecture in tests/ directory
• Review AWS EKS configuration changes for best practices
• Evaluate GitHub Actions workflow improvements and security
• Check Pulumi Infrastructure-as-Code patterns and conventions
• Assess security practices and credential management
• Validate deployment stage dependencies and stack references
• Review code quality and documentation updates
• Provide comprehensive review feedback

🟢 Strengths and Best Practices

1. Excellent Test Framework Architecture

• Robust ESC Token Management: The RefreshEscToken() and ExecuteWithRetry() utilities in tests/utils.go:114-220 are excellent for handling AWS credential expiration in long-running tests
• LIFO Cleanup Pattern: The CleanupStacksWithRetry() function (lines 235-270) properly destroys resources in reverse order, preventing dependency issues
• Environment Isolation: The TestEnvironment pattern provides proper test isolation with unique stack prefixes
• Comprehensive Service Validation: The service_validation.go module provides thorough end-to-end testing including user creation and organization management

2. Production-Ready Configuration Management

• Configurable Resource Protection: The protectResources configuration added across all EKS stages allows tests to disable protection while keeping production deployments protected by default
• Proper Stack References: Tests properly use opttest.UseAmbientBackend() for stack reference compatibility
• AWS Default Tags: Consistent tagging strategy with Purpose, AutoDelete, CreatedBy tags for resource management

3. Security Improvements

• GitHub Actions Version Tags: Excellent update from commit SHAs to version tags (e.g., @v4, @v1.2.0) for better security and maintainability
• Environment-based Secrets: Proper use of GitHub Actions environments for credential management
• Credential Validation: Environment variable validation functions prevent tests from running with missing credentials

🟡 Areas for Improvement

1. GitHub Actions Workflow Security

# In .github/workflows/main.yml:15
- uses: actions/checkout@ff7abcd0c3c05ccf6adc123a8cd1fd4fb30fb493 # v4

Issue: This still uses a commit SHA instead of the updated version tag pattern. Should be:

- uses: actions/checkout@v4

2. AWS Region Configuration

// In tests/aws_eks_ts_test.go:33
config := map[string]string{
    "aws:region": "us-east-1",
}

Recommendation: Consider making the AWS region configurable via environment variable to support multi-region testing and align with the ca-central-1 requirement mentioned in CLAUDE.md.

3. Test Timeout and Resource Limits

# In .github/workflows/test-aws-eks.yml:27
timeout-minutes: 120

Consideration: 120 minutes may be insufficient for the full 8-stage EKS deployment. Based on CLAUDE.md, EKS tests typically take ~2 hours. Consider increasing to 150 minutes to account for potential delays.

🔴 Critical Security Considerations

1. ESC Authentication Pattern

The CLAUDE.md mentions that AWS commands must use ESC authentication:

# Correct
esc run team-ce/default/aws go test -v -timeout=2h -tags aws ./...

However, the GitHub Actions workflow runs tests directly without ESC wrapper. Ensure the CI environment has proper AWS credentials configured.

2. Resource Cleanup Verification

The test framework has excellent cleanup utilities, but consider adding post-test verification to ensure no resources are left behind, especially important for the shared ca-central-1 account mentioned in CLAUDE.md.

📋 Deployment Pattern Validation

✅ Excellent Stage Dependencies

The test properly follows the 8-stage EKS deployment pattern:

1. 01-iam → 2. 02-networking → 3. 05-eks-cluster → 4. 10-cluster-svcs → 5. 15-state-policies-mgmt → 6. 20-database → 7. 25-insights → 8. 30-esc → 9. 90-pulumi-service

✅ Stack Reference Compatibility

Tests properly use shared backend for cross-stack references with UseAmbientBackend().

🛠️ Code Quality Assessment

Strengths:

• Go code follows proper error handling patterns with detailed logging
• Comprehensive test coverage with service validation
• Proper use of testify assertions
• Good separation of concerns between utilities and test logic

Minor Issues:

• Some unused functions have proper //nolint:unused comments
• File paths are properly handled with filepath.Join()
• Context handling is appropriate for Pulumi operations

📊 Documentation and Maintainability

✅ Excellent Documentation

• tests/README.md provides comprehensive usage instructions
• Clear build tag documentation for conditional compilation
• Proper Makefile targets for different test scenarios

Suggestion:

Consider adding a troubleshooting section to the README for common test failures (token expiration, resource quota limits, etc.).

🎯 Final Recommendations

1. Fix remaining commit SHA in main.yml workflow
2. Consider making AWS region configurable for multi-region testing
3. Increase GitHub Actions timeout to 150 minutes for safety
4. Add post-test resource verification to prevent leakage
5. Consider adding integration with the Makefile targets mentioned in CLAUDE.md

✅ Overall Assessment

This is an excellent pull request that significantly improves the testing infrastructure for Pulumi Self-Hosted installers. The token management utilities, cleanup patterns, and service validation framework are particularly well-designed. The GitHub Actions improvements enhance security and maintainability. With the minor fixes suggested above, this PR provides a robust foundation for continuous testing of the EKS deployment.

Recommendation: Approve with minor fixes

---

[claude]

Claude finished @rshade's task —— View job

---

Pull Request Review: AWS EKS Testing Framework

I've completed a comprehensive review of this PR that introduces AWS EKS testing infrastructure and GitHub Actions workflow improvements. Here's my detailed analysis:

Review Checklist:

• Read repository guidelines and understand codebase structure
• Analyze the new test framework architecture in tests/ directory
• Review AWS EKS configuration changes for best practices
• Evaluate GitHub Actions workflow improvements and security
• Check Pulumi Infrastructure-as-Code patterns and conventions
• Assess security practices and credential management
• Validate deployment stage dependencies and stack references
• Review code quality and documentation updates
• Provide comprehensive review feedback

🟢 Strengths and Best Practices

1. Excellent Test Framework Architecture

• Robust ESC Token Management: The RefreshEscToken() and ExecuteWithRetry() utilities in tests/utils.go:114-220 are excellent for handling AWS credential expiration in long-running tests
• LIFO Cleanup Pattern: The CleanupStacksWithRetry() function (lines 235-270) properly destroys resources in reverse order, preventing dependency issues
• Environment Isolation: The TestEnvironment pattern provides proper test isolation with unique stack prefixes
• Comprehensive Service Validation: The service_validation.go module provides thorough end-to-end testing including user creation and organization management

2. Production-Ready Configuration Management

• Configurable Resource Protection: The protectResources configuration added across all EKS stages allows tests to disable protection while keeping production deployments protected by default
• Proper Stack References: Tests properly use opttest.UseAmbientBackend() for stack reference compatibility
• AWS Default Tags: Consistent tagging strategy with Purpose, AutoDelete, CreatedBy tags for resource management

3. Security Improvements

• GitHub Actions Version Tags: Excellent update from commit SHAs to version tags (e.g., @v4, @v1.2.0) for better security and maintainability
• Environment-based Secrets: Proper use of GitHub Actions environments for credential management
• Credential Validation: Environment variable validation functions prevent tests from running with missing credentials

🟡 Areas for Improvement

1. GitHub Actions Workflow Security

# In .github/workflows/main.yml:15
- uses: actions/checkout@ff7abcd0c3c05ccf6adc123a8cd1fd4fb30fb493 # v4

Issue: This still uses a commit SHA instead of the updated version tag pattern. Should be:

- uses: actions/checkout@v4

2. AWS Region Configuration

// In tests/aws_eks_ts_test.go:33
config := map[string]string{
    "aws:region": "us-east-1",
}

Recommendation: Consider making the AWS region configurable via environment variable to support multi-region testing and align with the ca-central-1 requirement mentioned in CLAUDE.md.

3. Test Timeout and Resource Limits

# In .github/workflows/test-aws-eks.yml:27
timeout-minutes: 120

Consideration: 120 minutes may be insufficient for the full 8-stage EKS deployment. Based on CLAUDE.md, EKS tests typically take ~2 hours. Consider increasing to 150 minutes to account for potential delays.

🔴 Critical Security Considerations

1. ESC Authentication Pattern

The CLAUDE.md mentions that AWS commands must use ESC authentication:

# Correct
esc run team-ce/default/aws go test -v -timeout=2h -tags aws ./...

However, the GitHub Actions workflow runs tests directly without ESC wrapper. Ensure the CI environment has proper AWS credentials configured.

2. Resource Cleanup Verification

The test framework has excellent cleanup utilities, but consider adding post-test verification to ensure no resources are left behind, especially important for the shared ca-central-1 account mentioned in CLAUDE.md.

📋 Deployment Pattern Validation

✅ Excellent Stage Dependencies

The test properly follows the 8-stage EKS deployment pattern:

1. 01-iam → 2. 02-networking → 3. 05-eks-cluster → 4. 10-cluster-svcs → 5. 15-state-policies-mgmt → 6. 20-database → 7. 25-insights → 8. 30-esc → 9. 90-pulumi-service

✅ Stack Reference Compatibility

Tests properly use shared backend for cross-stack references with UseAmbientBackend().

🛠️ Code Quality Assessment

Strengths:

• Go code follows proper error handling patterns with detailed logging
• Comprehensive test coverage with service validation
• Proper use of testify assertions
• Good separation of concerns between utilities and test logic

Minor Issues:

• Some unused functions have proper //nolint:unused comments
• File paths are properly handled with filepath.Join()
• Context handling is appropriate for Pulumi operations

📊 Documentation and Maintainability

✅ Excellent Documentation

• tests/README.md provides comprehensive usage instructions
• Clear build tag documentation for conditional compilation
• Proper Makefile targets for different test scenarios

Suggestion:

Consider adding a troubleshooting section to the README for common test failures (token expiration, resource quota limits, etc.).

🎯 Final Recommendations

1. Fix remaining commit SHA in main.yml workflow
2. Consider making AWS region configurable for multi-region testing
3. Increase GitHub Actions timeout to 150 minutes for safety
4. Add post-test resource verification to prevent leakage
5. Consider adding integration with the Makefile targets mentioned in CLAUDE.md

✅ Overall Assessment

This is an excellent pull request that significantly improves the testing infrastructure for Pulumi Self-Hosted installers. The token management utilities, cleanup patterns, and service validation framework are particularly well-designed. The GitHub Actions improvements enhance security and maintainability. With the minor fixes suggested above, this PR provides a robust foundation for continuous testing of the EKS deployment.

Recommendation: Approve with minor fixes

---

[claude]

Claude finished @rshade's task —— View job

---

EKS-Hosted Directory Review

Reviewing changes in eks-hosted/** directory for AWS EKS 8-stage deployment architecture.

Review Checklist:

• Read repository guidelines and EKS deployment patterns
• Analyze stage dependency ordering (01→02→05→10→15→20→25→30→90)
• Review Pulumi configuration management across stages
• Validate EKS-specific security patterns
• Check TypeScript code quality and error handling
• Assess production readiness features
• Verify stack reference patterns
• Review resource protection configurations

---

🟢 Excellent Additions - Test-Enabling Features

1. Configurable Resource Protection Pattern

The most significant improvement is the consistent addition of protectResources configuration across all stages:

// eks-hosted/*/config.ts
const protectResources = pulumiConfig.getBoolean("protectResources") ?? true;

✅ Perfect Implementation:

• Secure by default: Production deployments protected (true by default)
• Test-friendly: Integration tests can disable protection (false)
• Consistent pattern: Applied across all modified stages (05, 15, 20, 30, 90)
• Proper application: Used in resource creation like eks-hosted/05-eks-cluster/index.ts:44

2. Excellent BYO Infrastructure Support

IAM stage (01-iam) demonstrates mature bring-your-own-infrastructure patterns:

// eks-hosted/01-iam/index.ts:20-26
if (config.eksServiceRoleName && config.eksInstanceRoleName && config.databaseMonitoringRoleArn) {
    // Use existing roles via aws.iam.Role.get()
} else {
    // Create new roles
}

✅ Strong Design:

• All-or-nothing validation: Prevents partial BYO configurations
• Consistent outputs: Same interface whether creating or using existing resources
• Clear documentation: Well-documented in Pulumi.README.yaml files

🟡 Areas for Improvement

1. Stack Reference Naming Inconsistencies

Found inconsistent stack reference patterns that could break inter-stage dependencies:

Issue in eks-hosted/20-database/config.ts:20:

// Inconsistent naming - should match project naming pattern
const clusterStackRef = new pulumi.StackReference(`${orgName}/selfhosted-05-ekscluster/${stackName}`);

Should be:

const clusterStackRef = new pulumi.StackReference(`${orgName}/selfhosted-05-eks-cluster/${stackName}`);

Issue in eks-hosted/90-pulumi-service/config.ts:15:
Same inconsistency - uses selfhosted-05-ekscluster instead of selfhosted-05-eks-cluster.

2. EKS Configuration Version Update

eks-hosted/05-eks-cluster/Pulumi.README.yaml:11 shows:

clusterVersion: "1.30"

While config.ts:37 defaults to "1.30.3". Consider documenting the specific patch version in the README for consistency.

🔴 Critical Security Considerations

1. Overly Broad IAM Permissions

Issue in eks-hosted/01-iam/index.ts:74-90:

const opensearchPolicy = new aws.iam.Policy("opensearchPolicy", {
    policy: {
        Statement: [{
            Action: ["es:*"],  // ⚠️ Too broad
            Effect: "Allow", 
            Resource: "*"      // ⚠️ Too broad
        }]
    }
});

Recommendation: Follow principle of least privilege:

Statement: [{
    Action: [
        "es:ESHttpGet",
        "es:ESHttpPost", 
        "es:ESHttpPut",
        "es:ESHttpDelete"
    ],
    Effect: "Allow",
    Resource: "arn:aws:es:${region}:${accountId}:domain/pulumi-insights/*"
}]

🛡️ Security Strengths

1. Proper Network Security

• Private subnets: EKS nodes deployed to private subnets (eks-hosted/05-eks-cluster/index.ts:80)
• No public IPs: nodeAssociatePublicIpAddress: false (line 22)
• Security group isolation: Proper security group references across stacks

2. Database Security

• Encryption at rest: storageEncrypted: true (eks-hosted/20-database/rds-db/index.ts:63)
• Strong password generation: 16-character passwords with special characters
• VPC isolation: Database subnets match EKS private subnets

3. Metadata Security

• IMDSv2 enforced: httpTokens: "required" prevents SSRF attacks
• Hop limit configured: httpPutResponseHopLimit: 2 limits token access

📋 Deployment Architecture Validation

✅ Proper Stage Dependencies

Stack references correctly implement the 8-stage dependency chain:

• 01-iam → 05-eks-cluster → 20-database → 90-pulumi-service
• Each stage properly references predecessors via pulumi.StackReference

✅ Configuration Management

• Consistent base naming: baseName propagated across all stages
• Proper secret handling: licenseKey marked as requireSecret()
• Environment flexibility: Configuration allows for different deployment sizes

🎯 Recommendations

1. Fix stack reference naming in database and Pulumi service configs
2. Tighten OpenSearch IAM policy to specific actions and resources
3. Align EKS version documentation between README and config defaults
4. Consider adding resource tags for better cost tracking and management

✅ Overall Assessment

This PR significantly enhances the EKS deployment's testability and production readiness. The configurable resource protection pattern is particularly well-implemented, enabling integration tests while maintaining security defaults. The BYO infrastructure support shows mature Infrastructure-as-Code patterns.

Key strengths:

• Excellent test framework integration with configurable protection
• Strong security fundamentals (encryption, network isolation, IMDSv2)
• Well-documented BYO infrastructure patterns
• Consistent configuration management across stages

Minor fixes needed:

• Stack reference naming consistency
• OpenSearch IAM policy scoping

Recommendation: Approve with minor fixes

---

[claude]

Claude finished @rshade's task —— View job

---

Pull Request Review: AWS EKS Testing Framework

I've completed a comprehensive review of the repository-wide changes in this PR, focusing on shared utilities, CI/CD improvements, and documentation updates as requested.

Review Summary:

• Read repository guidelines and understand codebase structure
• Analyze GitHub Actions workflow improvements and security
• Review new test framework architecture in tests/ directory
• Check Makefile and build automation additions
• Validate documentation updates (CLAUDE.md, README.md)
• Assess shared utilities and code quality patterns
• Check Pulumi IaC best practices in shared components
• Provide comprehensive review feedback

---

🟢 Strengths and Best Practices

1. Excellent Test Framework Architecture

• Robust ESC Token Management: The RefreshEscToken() and ExecuteWithRetry() utilities in tests/utils.go:115-220 provide excellent handling of AWS credential expiration during long-running tests
• LIFO Cleanup Pattern: Proper resource cleanup in reverse deployment order prevents dependency issues
• Environment Isolation: The TestEnvironment pattern provides test isolation with unique stack prefixes and temp directories
• Comprehensive Service Validation: End-to-end testing including user creation, organization management, and API endpoint validation

2. Production-Ready Documentation

• Comprehensive CLAUDE.md: Excellent coverage of testing patterns, token management, and platform-specific requirements
• Clear Build Tags: Proper Go build tag documentation (//go:build aws || all) for conditional compilation
• Detailed README: Well-structured test documentation with prerequisites, usage patterns, and troubleshooting guidance

3. Security Improvements

• GitHub Actions Modernization: Most workflows updated to use version tags (e.g., @v4, @v1.2.0) instead of commit SHAs
• Environment-based Secrets: Proper use of GitHub Actions environments for credential management
• Credential Validation: Environment variable validation functions prevent tests from running with missing credentials

4. Build Automation Excellence

• Quality-First Makefile: Comprehensive Go quality checks with fmt-go, vet-go, build-go, and lint-go targets
• Platform-Specific Tests: Individual test targets (test-aws-eks) with proper timeout configuration
• Dependency Management: Proper go mod download integration

---

🟡 Areas for Improvement

1. Inconsistent GitHub Actions Modernization

Issue: Some workflows still use commit SHAs instead of version tags:

# In .github/workflows/main.yml:15
- uses: actions/checkout@ff7abcd0c3c05ccf6adc123a8cd1fd4fb30fb493 # v4

# In .github/workflows/pr-test.yml:21
- uses: actions/checkout@ff7abcd0c3c05ccf6adc123a8cd1fd4fb30fb493 # v4

Recommendation: Update remaining workflows to use @v4 consistently for better security and maintainability.

2. Test Timeout Configuration

# In .github/workflows/test-aws-eks.yml:27
timeout-minutes: 120

Consideration: Based on CLAUDE.md documentation stating EKS tests take ~2 hours, consider increasing to 150 minutes to account for potential delays and ensure reliability.

3. AWS Region Configuration

The test framework hardcodes us-east-1 for AWS region. Consider making this configurable via environment variables to support multi-region testing and align with the ca-central-1 requirements mentioned in documentation.

---

🟢 Security Assessment

Credential Management

• ✅ Proper environment variable validation (checkAwsEnvVars, checkAzureEnvVars, checkGoogleEnvVars)
• ✅ GitHub Actions uses environment-specific secrets
• ✅ ESC authentication pattern properly documented
• ✅ No hardcoded credentials or secrets in code

Resource Management

• ✅ Configurable resource protection (protectResources: false for tests)
• ✅ Comprehensive cleanup utilities with retry logic
• ✅ AWS default tags for resource tracking and cleanup

---

📋 Code Quality Assessment

Go Code Standards

• ✅ Proper error handling patterns with detailed logging
• ✅ Appropriate use of testify assertions
• ✅ Good separation of concerns between utilities and test logic
• ✅ Proper //nolint:unused comments for conditionally compiled functions
• ✅ Context handling appropriate for Pulumi operations

Test Utilities

• ✅ Token Expiration Handling: IsTokenExpiredError() properly detects various AWS token errors
• ✅ Retry Logic: ExecuteWithRetry() provides robust operation retry with token refresh
• ✅ File Operations: CopyFile() includes proper error handling and permission preservation
• ✅ Environment Isolation: TestEnvironment provides complete test isolation

---

🎯 Final Recommendations

Critical (Must Fix)

1. Complete GitHub Actions modernization: Update remaining commit SHAs to version tags in main.yml and pr-test.yml

Important (Should Fix)

2. Increase test timeout: Consider 150 minutes for EKS tests to ensure reliability
3. Make AWS region configurable: Support multi-region testing via environment variables
4. Add missing golangci-lint: The Makefile references golangci-lint but it may not be available in CI

Nice to Have

5. Add troubleshooting section to test README for common failure scenarios
6. Consider post-test resource verification to prevent resource leakage
7. Add integration with ESC authentication in GitHub Actions workflows

---

✅ Overall Assessment

This is an excellent pull request that significantly enhances the repository's testing infrastructure and CI/CD practices. The test framework demonstrates sophisticated understanding of Pulumi deployment patterns, AWS token management, and production-ready testing practices.

Key Highlights:

• Token Management: Solves the critical challenge of 60+ minute test reliability with proper ESC token refresh
• Resource Cleanup: LIFO cleanup pattern prevents dependency issues and resource quota exhaustion
• Documentation: Comprehensive guidance for developers and CI/CD systems
• Security: Modern GitHub Actions practices and proper credential management

Recommendation: ✅ Approve with minor fixes

The identified issues are primarily cosmetic (inconsistent SHA usage) and configuration improvements. The core architecture and implementation are production-ready and follow Pulumi best practices.

---