Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stacks: values generation #3914

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
Open

Stacks: values generation #3914

wants to merge 12 commits into from

Conversation

denis256
Copy link
Member

@denis256 denis256 commented Feb 20, 2025

Description

  • Added generation of terragrunt.values.hcl during stack generate
  • Added loading of terragrunt.values.hcl as values during execution
  • Replaced unit.values with values

Demo:

stack-values

Fixes #3911.

TODOs

Read the Gruntwork contribution guidelines.

  • Update the docs.
  • Run the relevant tests successfully, including pre-commit checks.
  • Ensure any 3rd party code adheres with our license policy or delete this line if its not applicable.
  • Include release notes. If this PR is backward incompatible, include a migration guide.

Release Notes (draft)

Added / Removed / Updated [X].

Migration Guide

Variable unit.values was deprecated and replaced with values, use values.* to reference unit values.

Summary by CodeRabbit

  • New Features

    • Enhanced stack configuration management with support for unit-specific values, enabling automatic generation and reading of unit configuration files.
    • Improved configuration parsing to support new experimental stack features.
  • Refactor

    • Simplified stack command execution by removing legacy steps for populating values.
    • Streamlined configuration path resolution and standardized directory permission settings.
    • Updated input reference syntax for improved clarity.
  • Tests

    • Added integration tests to validate stack application behavior in subdirectory scenarios.
    • Introduced a new test function to verify unit values handling in the stack.

Copy link

vercel bot commented Feb 20, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
terragrunt-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Feb 21, 2025 6:01pm

Copy link
Contributor

coderabbitai bot commented Feb 20, 2025

Walkthrough

This pull request simplifies stack command execution by removing the populateStackValues function and its associated error handling. Instead, the configuration path is set directly by combining the working directory with a default filename. The changes introduce functionality to write and read unit-specific values files while updating permissions and evaluation contexts. Adjustments in config modules refactor how unit values are processed, and the options structure is streamlined by removing unused fields and methods. A new integration test verifies stack application in a subdirectory context.

Changes

File(s) Change Summary
cli/commands/stack/action.go Removed populateStackValues function and its calls from RunGenerate, Run, and RunOutput. Renamed dirPerm to defaultPerms.
cli/commands/stack/generate.go, cli/commands/stack/output.go Updated configuration path to use opts.WorkingDir and default file; in generate, updated directory permissions and added unit values file generation via config.WriteUnitValues.
config/config.go, config/config_helpers.go, config/config_partial.go, config/locals.go, config/parsing_context.go Added constant StackValuesFile, removed MetadataUnit, and modified evaluation context to use MetadataValues; integrated reading unit values in parsing functions and added a new field/method in ParsingContext.
config/stack.go Added WriteUnitValues and ReadUnitValues functions along with constants (unitValuesFile, defaultPerms) to manage unit-specific HCL value files.
options/options.go Removed the StackValues struct and its related methods and field from TerragruntOptions to reduce complexity.
test/fixtures/stacks/unit-values/units/app/terragrunt.hcl Updated references in locals and inputs blocks from unit.values.* to direct values.* usage.
test/integration_stacks_test.go Added new integration test TestStacksUnitValuesRunInApp1 to validate stack application in a subdirectory context.

Sequence Diagram(s)

sequenceDiagram
  participant U as User
  participant CLI as CLI Command
  participant GS as generateStack
  participant PS as processStackFile
  participant WUV as WriteUnitValues
  participant RUV as ReadUnitValues

  U->>CLI: Execute stack command
  CLI->>GS: Call generateStack
  GS->>GS: Set TerragruntStackConfigPath (WorkingDir + defaultStackFile)
  GS->>PS: Process stack configuration
  PS->>WUV: For each unit, generate unit values file
  WUV-->>PS: Unit values written
  Note over CLI,RUV: During config parsing
  CLI->>RUV: Read unit values from file
  RUV-->>CLI: Return unit values
Loading

Possibly related PRs

  • Stack: stack values #3877: Introduced the populateStackValues function that is now removed in this PR.
  • Stacks: output #3796: Focused on the removal of stack-related functions, indicating a significant restructuring that relates to this PR.

Suggested reviewers

  • levkohimins
  • yhakbar

Poem

In code we trim the extra weight,
Functions removed to make it great.
Unit values now take their place,
Clean and clear in every space.
Cheers to changes, light as air! 🚀😄

✨ Finishing Touches
  • 📝 Generate Docstrings (Beta)

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Beta)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (4)
cli/commands/stack/generate.go (1)

82-85: Consider adding error context for better debugging.

When generating the unit values file, consider wrapping the error with more context about which unit failed.

 // generate unit values file
 if err := config.WriteUnitValues(opts, unit, dest); err != nil {
-    return errors.New(err)
+    return errors.New(fmt.Errorf("failed to generate values file for unit %s: %w", unit.Name, err))
 }
cli/commands/stack/output.go (1)

24-24: Consider extracting path construction to a shared function.

This path construction logic is duplicated in both generate.go and output.go. Consider extracting it to a shared function to maintain consistency.

+// getDefaultStackConfigPath returns the default stack configuration path
+func getDefaultStackConfigPath(workingDir string) string {
+    return filepath.Join(workingDir, defaultStackFile)
+}

 func generateOutput(ctx context.Context, opts *options.TerragruntOptions) (map[string]map[string]cty.Value, error) {
     opts.Logger.Debugf("Generating output from %s", opts.TerragruntStackConfigPath)
-    opts.TerragruntStackConfigPath = filepath.Join(opts.WorkingDir, defaultStackFile)
+    opts.TerragruntStackConfigPath = getDefaultStackConfigPath(opts.WorkingDir)
config/stack.go (2)

96-118: LGTM! Consider enhancing error handling.

The function correctly handles the generation of unit values, with good nil checks and debug logging. However, we could improve the error handling by wrapping the file write error with more context.

-	if err := os.WriteFile(filePath, file.Bytes(), defaultPerms); err != nil {
-		return err
+	if err := os.WriteFile(filePath, file.Bytes(), defaultPerms); err != nil {
+		return fmt.Errorf("failed to write unit values file %s: %w", filePath, err)

120-150: Maintain consistent error handling style.

The function handles the reading of unit values well, but the error handling style varies between direct returns and wrapping with errors.New().

-		return nil, errors.New(err)
+		return nil, fmt.Errorf("failed to parse unit values file %s: %w", filePath, err)
-		return nil, errors.New(err)
+		return nil, fmt.Errorf("failed to evaluate parsing context for %s: %w", filePath, err)
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 5d2b955 and 7f73714.

📒 Files selected for processing (12)
  • cli/commands/stack/action.go (1 hunks)
  • cli/commands/stack/generate.go (3 hunks)
  • cli/commands/stack/output.go (2 hunks)
  • config/config.go (2 hunks)
  • config/config_helpers.go (1 hunks)
  • config/config_partial.go (2 hunks)
  • config/locals.go (1 hunks)
  • config/parsing_context.go (2 hunks)
  • config/stack.go (4 hunks)
  • options/options.go (0 hunks)
  • test/fixtures/stacks/unit-values/units/app/terragrunt.hcl (1 hunks)
  • test/integration_stacks_test.go (1 hunks)
💤 Files with no reviewable changes (1)
  • options/options.go
⏰ Context from checks skipped due to timeout of 90000ms (2)
  • GitHub Check: build-and-test
  • GitHub Check: Pull Request has non-contributor approval
🔇 Additional comments (10)
cli/commands/stack/action.go (1)

20-20: LGTM! Clear and descriptive constant name.

The rename from dirPerm to defaultPerms better reflects its purpose as the default permissions for directory creation.

cli/commands/stack/generate.go (1)

25-25:

❓ Verification inconclusive

Verify path construction across different OS environments.

While joining paths is correct, ensure this works correctly on Windows systems too.


🏁 Script executed:

#!/bin/bash
# Check if the code handles Windows paths correctly
ast-grep --pattern 'filepath.Join($_)'

Length of output: 40


Attention: Please Verify Windows Environment Handling

The change using filepath.Join(opts.WorkingDir, defaultStackFile) in cli/commands/stack/generate.go looks correct since it relies on Go's standard library, which generally manages cross-platform path separators properly. However, the automated check (using ast-grep) didn’t yield explicit confirmation of Windows-specific behavior. I recommend manually verifying that this path construction works as intended on Windows—ideally by running tests in a Windows environment or using a simulator. This extra step will help confirm that error handling and OS-specific formatting are solid.

config/parsing_context.go (2)

30-32: LGTM! Well-documented field addition.

The new Values field is properly documented and follows the existing pattern of optional context values.


78-81: LGTM! Consistent method implementation.

The WithValues method follows the established builder pattern used throughout the file.

config/locals.go (1)

185-187: LGTM! Clean replacement of metadata handling.

The switch from MetadataUnit to MetadataValues is clean and maintains consistency with the existing code structure.

config/config_partial.go (1)

351-360: LGTM! Clean integration of unit values handling.

The addition of unit values handling for the Stacks experiment is well-integrated with the existing parsing logic and maintains proper error handling.

config/config_helpers.go (1)

213-215: LGTM! Clean implementation of the values context.

The change properly handles the new values structure while maintaining nil safety checks.

config/config.go (2)

45-45: LGTM! Clear and descriptive constant definition.

The constant name and value are self-explanatory and follow the existing naming conventions.


897-906: LGTM! Well-structured integration of unit values.

The code properly:

  • Checks for the Stacks experiment flag
  • Handles errors from ReadUnitValues
  • Updates the context with the new values
test/fixtures/stacks/unit-values/units/app/terragrunt.hcl (1)

3-3: LGTM! Consistent update of value references.

The changes correctly update the value references to use the new structure, maintaining consistency across both locals and inputs blocks.

Also applies to: 7-8

Comment on lines 365 to 377
func TestStacksUnitValuesRunInApp1(t *testing.T) {
t.Parallel()

helpers.CleanupTerraformFolder(t, testFixtureStacksUnitValues)
tmpEnvPath := helpers.CopyEnvironment(t, testFixtureStacksUnitValues)
rootPath := util.JoinPath(tmpEnvPath, testFixtureStacksUnitValues)

helpers.RunTerragrunt(t, "terragrunt stack run apply --experiment stacks --terragrunt-non-interactive --terragrunt-working-dir "+rootPath)

// run apply in generated app1 directory
app1Path := util.JoinPath(rootPath, ".terragrunt-stack", "app1")
helpers.RunTerragrunt(t, "terragrunt apply --experiment stacks --terragrunt-non-interactive --terragrunt-working-dir "+app1Path)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Add assertions to verify the test outcome.

While the test executes the stack commands correctly, it doesn't verify the actual outcome of the operations. Consider adding assertions to validate the expected state after the commands are run.

 	app1Path := util.JoinPath(rootPath, ".terragrunt-stack", "app1")
 	helpers.RunTerragrunt(t, "terragrunt apply --experiment stacks --terragrunt-non-interactive --terragrunt-working-dir "+app1Path)
+
+	// Verify the expected outcomes
+	valuesPath := filepath.Join(app1Path, "terragrunt.values.hcl")
+	assert.FileExists(t, valuesPath)
+
+	// Verify the values file content
+	content, err := os.ReadFile(valuesPath)
+	require.NoError(t, err)
+	assert.Contains(t, string(content), "deployment = \"app1\"")
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
func TestStacksUnitValuesRunInApp1(t *testing.T) {
t.Parallel()
helpers.CleanupTerraformFolder(t, testFixtureStacksUnitValues)
tmpEnvPath := helpers.CopyEnvironment(t, testFixtureStacksUnitValues)
rootPath := util.JoinPath(tmpEnvPath, testFixtureStacksUnitValues)
helpers.RunTerragrunt(t, "terragrunt stack run apply --experiment stacks --terragrunt-non-interactive --terragrunt-working-dir "+rootPath)
// run apply in generated app1 directory
app1Path := util.JoinPath(rootPath, ".terragrunt-stack", "app1")
helpers.RunTerragrunt(t, "terragrunt apply --experiment stacks --terragrunt-non-interactive --terragrunt-working-dir "+app1Path)
}
func TestStacksUnitValuesRunInApp1(t *testing.T) {
t.Parallel()
helpers.CleanupTerraformFolder(t, testFixtureStacksUnitValues)
tmpEnvPath := helpers.CopyEnvironment(t, testFixtureStacksUnitValues)
rootPath := util.JoinPath(tmpEnvPath, testFixtureStacksUnitValues)
helpers.RunTerragrunt(t, "terragrunt stack run apply --experiment stacks --terragrunt-non-interactive --terragrunt-working-dir "+rootPath)
// run apply in generated app1 directory
app1Path := util.JoinPath(rootPath, ".terragrunt-stack", "app1")
helpers.RunTerragrunt(t, "terragrunt apply --experiment stacks --terragrunt-non-interactive --terragrunt-working-dir "+app1Path)
// Verify the expected outcomes
valuesPath := filepath.Join(app1Path, "terragrunt.values.hcl")
assert.FileExists(t, valuesPath)
// Verify the values file content
content, err := os.ReadFile(valuesPath)
require.NoError(t, err)
assert.Contains(t, string(content), "deployment = \"app1\"")
}

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (3)
config/stack.go (3)

24-25: Consider using more restrictive file permissions.

The defaultPerms value of 0755 might be too permissive for a values file that could contain sensitive information. Consider using 0600 (user read/write only) or 0640 (user read/write, group read only) instead.

-	defaultPerms   = 0755
+	defaultPerms   = 0600

116-118: Minor style tweak: Add line break before range.

There's a minor style issue with the cuddle assignment. Let's add a line break before the range statement to match Go's style conventions.

	})
+
	for key, val := range unit.Values.AsValueMap() {
		body.SetAttributeValue(key, val)
	}
🧰 Tools
🪛 golangci-lint (1.62.2)

116-116: only one cuddle assignment allowed before range statement

(wsl)


127-157: Add input validation for directory path.

The function looks solid overall, but let's add some input validation to ensure the directory path is valid.

 func ReadUnitValues(ctx context.Context, opts *options.TerragruntOptions, unitDirectory string) (*cty.Value, error) {
+	if unitDirectory == "" {
+		return nil, fmt.Errorf("unit directory path cannot be empty")
+	}
+
 	filePath := filepath.Join(unitDirectory, unitValuesFile)
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 5268b6c and c58a7b6.

📒 Files selected for processing (1)
  • config/stack.go (4 hunks)
🧰 Additional context used
🪛 golangci-lint (1.62.2)
config/stack.go

116-116: only one cuddle assignment allowed before range statement

(wsl)

⏰ Context from checks skipped due to timeout of 90000ms (2)
  • GitHub Check: build-and-test
  • GitHub Check: Pull Request has non-contributor approval
🔇 Additional comments (2)
config/stack.go (2)

98-125: Well-structured implementation with clear error handling!

The function is well-implemented with:

  • Early return for nil values
  • Clear debug logging
  • Helpful auto-generated comment in the file
  • Proper error handling
🧰 Tools
🪛 golangci-lint (1.62.2)

116-116: only one cuddle assignment allowed before range statement

(wsl)


170-171: Nice optimization with map pre-allocation! 👍

Pre-allocating the maps with the known capacity is a good practice that helps avoid unnecessary map growth and rehashing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Adjust Design for Stack Values
1 participant