Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
ecb150c
fix: prevent path duplication when using absolute paths in component …
osterman Sep 27, 2025
c3e9dc6
test: add Windows-specific absolute path test case
osterman Sep 27, 2025
0b48ee6
refactor: eliminate duplicate path joining logic
osterman Sep 27, 2025
99c4955
docs: add architecture decision for path construction vs validation
osterman Sep 27, 2025
da769e7
test: add comprehensive tests for JoinPath function
osterman Sep 27, 2025
80d7cb1
test: add comprehensive Windows and Unix path edge cases
osterman Sep 27, 2025
21f4111
docs: convert architecture decision to PRD format
osterman Sep 27, 2025
69357e2
fix: use JoinPath in sandbox to handle absolute component paths
osterman Sep 27, 2025
57bbc2b
test: add specific test for absolute component paths in sandbox
osterman Sep 27, 2025
4a8a44b
test: fix error handling and path expectations in tests
osterman Sep 27, 2025
e0c2d3e
fix: resolve path handling issues in JoinPath and component resolution
osterman Sep 28, 2025
d5c9564
fix: resolve linting issues and improve code quality
osterman Sep 28, 2025
93a243b
fix: normalize Windows path handling to match Go standards
osterman Sep 28, 2025
e5073d9
fix: resolve remaining Windows test failures
osterman Sep 28, 2025
49c8bee
fix: skip URL-like path test on Windows
osterman Sep 28, 2025
05dbf00
docs: add Windows path handling documentation and tests
osterman Sep 28, 2025
8d1e3e8
fix: improve path handling and test clarity
osterman Sep 28, 2025
015b2e9
Merge branch 'main' into fix-component-dir
osterman Sep 28, 2025
0ebfc27
refactor: use actual AtmosConfigAbsolutePaths instead of simulating it
osterman Sep 28, 2025
f478806
Merge branch 'fix-component-dir' of https://github.com/cloudposse/atm…
osterman Sep 28, 2025
aadd4a7
fix: improve UNC path handling and fix linting issues
osterman Sep 28, 2025
91b9eee
fix: use platform-appropriate path separators in Windows test
osterman Sep 29, 2025
24a082a
Merge branch 'main' into fix-component-dir
osterman Sep 29, 2025
d3dc4ad
fix: replace unsafe hardcoded Windows paths with safe temp-based paths
osterman Sep 29, 2025
07ca3cd
fix: comprehensive Windows path handling improvements
osterman Sep 29, 2025
c082d1f
fix: clean up test structure and add atmos binary to gitignore
osterman Sep 29, 2025
6348873
refactor: rename JoinAbsolutePathWithPath to JoinPathAndValidate and …
osterman Sep 29, 2025
03f7d6c
Merge branch 'main' into fix-component-dir
osterman Sep 29, 2025
4db3da0
docs: update PRD dates to reflect 2025 authorship
osterman Sep 29, 2025
a0ff4fd
Merge branch 'main' into fix-component-dir
aknysh Sep 29, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,9 @@
**/kubeconfig.yaml
**/*.planfile.json

# Atmos binary
atmos

# Module directory
.terraform
**/.idea
Expand Down
8 changes: 4 additions & 4 deletions docs/prd/command-merging.md
Original file line number Diff line number Diff line change
Expand Up @@ -233,10 +233,10 @@ None - all requirements have been implemented and tested.

| Date | Decision | Rationale |
|------|----------|-----------|
| 2024-01-26 | Use name-based override instead of deep merge | Simpler mental model, predictable behavior |
| 2024-01-26 | Process `.atmos.d/` before explicit imports | Follows precedence principle: more specific overrides less specific |
| 2024-01-26 | Support glob patterns in imports | Enables modular command organization |
| 2024-01-26 | Same code path for all import types | Ensures consistent behavior, easier maintenance |
| 2025-09-26 | Use name-based override instead of deep merge | Simpler mental model, predictable behavior |
| 2025-09-26 | Process `.atmos.d/` before explicit imports | Follows precedence principle: more specific overrides less specific |
| 2025-09-26 | Support glob patterns in imports | Enables modular command organization |
| 2025-09-26 | Same code path for all import types | Ensures consistent behavior, easier maintenance |

## References

Expand Down
293 changes: 293 additions & 0 deletions docs/prd/path-construction-vs-validation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,293 @@
# PRD: Path Construction vs Validation Architecture

## Document Control
- **PRD Number**: PRD-2025-001
- **Author**: Engineering Team
- **Date Created**: 2025-09-27
- **Last Updated**: 2025-09-27
- **Status**: Implemented
- **Related Issues**: #1512, #1535

## Executive Summary

This PRD documents the architectural decision to separate path construction from filesystem validation in Atmos. This separation improves testability, maintainability, and follows the Single Responsibility Principle while fixing a critical path duplication bug that affected GitHub Actions workflows.

## Problem Statement

### Background
When component paths were configured with absolute paths in Atmos, the `filepath.Join()` function on Unix systems incorrectly handled two absolute paths, causing path duplication. This manifested as broken GitHub Actions pipelines with duplicated paths like:
```text
/home/runner/_work/infrastructure/infrastructure/home/runner/_work/infrastructure/infrastructure/atmos/components/terraform
```

### Root Cause Analysis
The issue stemmed from mixing path construction logic with filesystem validation in the `JoinAbsolutePathWithPath` function, which:
1. Performed path string manipulation
2. Executed filesystem checks (`os.Stat`)
3. Failed in unit tests when paths didn't exist
4. Violated the Single Responsibility Principle

### Impact
- **Users Affected**: All users upgrading from v1.191.0 to v1.192.0
- **Severity**: High - Broken CI/CD pipelines
- **Platforms**: Primarily Unix/Linux, with potential Windows issues

## Requirements

### Functional Requirements

#### FR1: Path Construction
- **FR1.1**: Pure functions that manipulate path strings without I/O operations
- **FR1.2**: Handle absolute paths correctly on all platforms (Windows, Unix, macOS)
- **FR1.3**: Prevent path duplication when joining two absolute paths
- **FR1.4**: Support all path formats:
- Unix: `/home/user/project`
- Windows: `C:\Users\project`
- UNC: `\\server\share\project`
- Long paths: `\\?\C:\very\long\path`

#### FR2: Path Validation
- **FR2.1**: Separate functions for filesystem checks
- **FR2.2**: Validation only when explicitly needed
- **FR2.3**: Clear error messages when paths don't exist

### Non-Functional Requirements

#### NFR1: Testability
- **NFR1.1**: Path construction functions must be testable without filesystem mocks
- **NFR1.2**: >80% test coverage for path operations
- **NFR1.3**: Tests must run on all platforms without modification

#### NFR2: Performance
- **NFR2.1**: No unnecessary filesystem operations during path construction
- **NFR2.2**: Lazy validation - only check existence when required

#### NFR3: Maintainability
- **NFR3.1**: Clear separation of concerns
- **NFR3.2**: Self-documenting function names
- **NFR3.3**: Consistent patterns across the codebase

## Design

### Architecture Overview

```text
┌─────────────────────────────────────────┐
│ Application Layer │
│ (Commands: terraform, helmfile, etc.) │
└────────────┬────────────────────────────┘
│
▼
┌─────────────────────────────────────────┐
│ Path Construction Layer │
│ (Pure Functions - No I/O) │
│ ┌─────────────────────────────────┐ │
│ │ JoinPath(base, provided) │ │
│ │ buildComponentPath(...) │ │
│ │ GetComponentPath(...) │ │
│ └─────────────────────────────────┘ │
└────────────┬────────────────────────────┘
│
▼
┌─────────────────────────────────────────┐
│ Path Validation Layer │
│ (I/O Operations - Mockable) │
│ ┌─────────────────────────────────┐ │
│ │ IsDirectory(path) │ │
│ │ FileExists(path) │ │
│ │ FileOrDirExists(path) │ │
│ └─────────────────────────────────┘ │
└──────────────────────────────────────────┘
```

### Core Components

#### 1. Path Construction Functions (Pure)

```go
// JoinPath - Pure function for path manipulation
// No filesystem checks, no side effects
func JoinPath(basePath, providedPath string) string {
if filepath.IsAbs(providedPath) {
return providedPath
}
return filepath.Join(basePath, providedPath)
}
```

**Characteristics:**
- Deterministic output
- No side effects
- Platform-agnostic using `filepath` package
- Easily testable

#### 2. Path Validation Functions (I/O)

```go
// IsDirectory - I/O function that checks filesystem
func IsDirectory(path string) (bool, error) {
fileInfo, err := os.Stat(path)
if err != nil {
return false, err
}
return fileInfo.IsDir(), nil
}
```

**Characteristics:**
- Performs actual filesystem operations
- Returns errors for missing paths
- Used at application boundaries
- Can be mocked if needed

### Usage Pattern

```go
// Step 1: Construct the path (pure, no I/O)
componentPath := GetComponentPath(config, "terraform", prefix, component)

// Step 2: Validate when needed (I/O operation)
if exists, err := IsDirectory(componentPath); !exists {
return fmt.Errorf("component not found: %s", componentPath)
}

// Step 3: Use the validated path
return ExecuteTerraformCommand(componentPath, args)
```

## Implementation

### Phase 1: Core Functions (Completed)
- [x] Implement `JoinPath` utility function
- [x] Refactor `buildComponentPath` to use `JoinPath`
- [x] Update `atmosConfigAbsolutePaths` to use `JoinPath`

### Phase 2: Testing (Completed)
- [x] Unit tests for path construction (no mocks needed)
- [x] Cross-platform path handling tests (65+ scenarios)
- [x] Windows and Unix edge case tests (comprehensive coverage)
- [x] Integration tests with real filesystem
Comment thread
coderabbitai[bot] marked this conversation as resolved.

### Phase 3: Migration (Completed)
- [x] Update all path joining to use new utilities
- [x] Remove duplicate path logic
- [x] Ensure backward compatibility

## Testing Strategy

### Unit Tests
- Test path construction with various inputs
- No filesystem mocks required
- Cross-platform test cases

### Integration Tests
- Test full command flow with real filesystem
- Use test fixtures in `tests/` directory
- Validate actual file operations

### Test Coverage Matrix

| Scenario | Windows | Unix | macOS |
|----------|---------|------|-------|
| Absolute paths | ✅ | ✅ | ✅ |
| Relative paths | ✅ | ✅ | ✅ |
| UNC paths | ✅ | N/A | N/A |
| Drive letters | ✅ | N/A | N/A |
| Hidden files | ✅ | ✅ | ✅ |
| Special chars | ✅ | ✅ | ✅ |
| Path navigation | ✅ | ✅ | ✅ |

## Alternatives Considered

### Alternative 1: Filesystem Interface with Dependency Injection

```go
type FileSystem interface {
Stat(path string) (os.FileInfo, error)
ReadFile(path string) ([]byte, error)
}
```

**Pros:**
- Full mockability
- Complete control in tests

**Cons:**
- Over-engineering for current needs
- Increases complexity
- Requires refactoring all file operations

**Decision:** Rejected - Current approach provides sufficient testability

### Alternative 2: Keep Mixed Responsibilities

**Pros:**
- No refactoring needed
- Single function call

**Cons:**
- Poor testability
- Violates SRP
- Continued test failures

**Decision:** Rejected - Causes test failures and maintenance issues

## Success Metrics

1. **Test Coverage**: >80% coverage for path operations ✅
2. **Bug Resolution**: No path duplication in any scenario ✅
3. **Cross-Platform**: Tests pass on Windows, Linux, macOS ✅
4. **Performance**: No unnecessary filesystem operations ✅
5. **Maintainability**: Clear separation of concerns ✅

## Security Considerations

1. **Path Traversal**: Functions don't prevent `../` navigation - this is intentional as some workflows require it
2. **Symbolic Links**: Not resolved during path construction - resolution happens at validation
3. **Permissions**: Validation functions return appropriate errors for permission issues

## Migration Guide

### For Atmos Core Team

1. Use `JoinPath` for all new path joining operations
2. Call validation functions only at command boundaries
3. Keep path construction and validation separate

### For Plugin Developers

No changes required - external API remains the same

## Rollout Plan

1. **v1.193.0**: Include fix with separated path logic
2. **Documentation**: Update contributor guidelines
3. **Monitoring**: Watch for path-related issues in GitHub Issues

## Appendix

### A. Function Inventory

#### Path Construction (Pure)
- `JoinPath(basePath, providedPath string) string`
- `buildComponentPath(basePath, folderPrefix, component string) string`
- `GetComponentPath(config, type, prefix, component) (string, error)`

#### Path Validation (I/O)
- `IsDirectory(path string) (bool, error)`
- `FileExists(path string) bool`
- `FileOrDirExists(path string) bool`

### B. References

- PR #1512: Introduction of regression
- PR #1535: Implementation of fix
- Go filepath package documentation
- Single Responsibility Principle

### C. Glossary

- **Pure Function**: Function with no side effects, deterministic output
- **I/O Operation**: Operation that interacts with filesystem
- **Path Duplication**: Bug where absolute paths were incorrectly concatenated
- **SRP**: Single Responsibility Principle
6 changes: 6 additions & 0 deletions errors/errors.go
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,12 @@ var (
ErrReadFile = errors.New("error reading file")
ErrInvalidFlag = errors.New("invalid flag")

// File and URL handling errors.
ErrInvalidPagerCommand = errors.New("invalid pager command")
ErrEmptyURL = errors.New("empty URL provided")
ErrInvalidURL = errors.New("invalid URL")
ErrFailedToFindImport = errors.New("failed to find import")

ErrMissingStack = errors.New("stack is required; specify it on the command line using the flag `--stack <stack>` (shorthand `-s`)")
ErrInvalidComponent = errors.New("invalid component")
ErrAbstractComponentCantBeProvisioned = errors.New("abstract component cannot be provisioned")
Expand Down
2 changes: 1 addition & 1 deletion internal/exec/describe_affected_utils.go
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ func executeDescribeAffected(
atmosConfig.TerraformDirAbsolutePath = filepath.Join(remoteRepoFileSystemPath, basePath, atmosConfig.Components.Terraform.BasePath)
atmosConfig.HelmfileDirAbsolutePath = filepath.Join(remoteRepoFileSystemPath, basePath, atmosConfig.Components.Helmfile.BasePath)
atmosConfig.PackerDirAbsolutePath = filepath.Join(remoteRepoFileSystemPath, basePath, atmosConfig.Components.Packer.BasePath)
atmosConfig.StackConfigFilesAbsolutePaths, err = u.JoinAbsolutePathWithPaths(
atmosConfig.StackConfigFilesAbsolutePaths, err = u.JoinPaths(
filepath.Join(remoteRepoFileSystemPath, basePath, atmosConfig.Stacks.BasePath),
atmosConfig.StackConfigFilesRelativePaths,
)
Expand Down
Loading
Loading