Skip to content

feat: Support template extraction from generated code#491

Merged
kingston merged 8 commits into
mainfrom
kingston/eng-498-build-reverse-generator-architecture
Apr 10, 2025
Merged

feat: Support template extraction from generated code#491
kingston merged 8 commits into
mainfrom
kingston/eng-498-build-reverse-generator-architecture

Conversation

@kingston

@kingston kingston commented Apr 10, 2025

Copy link
Copy Markdown
Collaborator

Summary by CodeRabbit

  • New Features

    • Introduced a CLI command for extracting templates.
    • Enhanced dynamic README generation using templating.
    • Added a feature flag enabling template extractor functionality, now accessible via a new settings page in the web app.
    • Added a new environment variable for enabling template extractor in development.
    • Introduced a new React component for managing template extractor settings.
    • Added new export statements for template extraction utilities and types.
  • Documentation

    • Updated guidance on test setups and configuration for improved clarity.
    • Added new documentation for template extraction functionality.
  • Chores/Improvements

    • Refined build and script configurations to streamline development.
    • Enhanced file generation and JSON formatting processes.
    • Removed unnecessary dependencies to simplify the project.
  • Tests

    • Expanded test coverage for template extraction and metadata handling.
    • Added unit tests for new template extraction functionalities.

@linear

linear Bot commented Apr 10, 2025

Copy link
Copy Markdown
ENG-498 Build template extractor architecture

Make it possible to extract changes from the target codebase back into the generator templates to allow for more efficient generator updates.

@changeset-bot

changeset-bot Bot commented Apr 10, 2025

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: e101806

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 14 packages
Name Type
@halfdomelabs/sync Minor
@halfdomelabs/project-builder-server Patch
@halfdomelabs/project-builder-cli Patch
@halfdomelabs/project-builder-lib Patch
@halfdomelabs/project-builder-web Patch
@halfdomelabs/fastify-generators Patch
@halfdomelabs/tools Patch
@halfdomelabs/utils Patch
@halfdomelabs/core-generators Patch
@halfdomelabs/project-builder-common Patch
@halfdomelabs/project-builder-test Patch
@halfdomelabs/react-generators Patch
@halfdomelabs/baseplate-plugin-storage Major
@halfdomelabs/create-project Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@coderabbitai

coderabbitai Bot commented Apr 10, 2025

Copy link
Copy Markdown

Walkthrough

A new file documents version updates for packages under the @halfdomelabs scope, indicating minor and patch updates across several packages and the introduction of template extractor functionality. Documentation for Vitest testing has been clarified to emphasize the necessary imports. The package.json files have been updated to include a new script for extracting templates and modifications to existing scripts. The README generation process has been enhanced to use dynamic templating. New CLI commands have been added for build and extraction processes, and extensive changes have been made to support template extraction in various packages, including the addition of new components and utilities.

Changes

File(s) Change Summary
.changeset/early-heads-post.md New file documenting version updates for packages and support for template extractor functionality.
.cursor/rules/writing-vitest-tests.mdc Documentation update emphasizing explicit import of Vitest globals (expect, describe).
package.json
packages/project-builder-cli/package.json
Added new script "extract:templates" and updated dev/start scripts to use a custom tsconfig.
packages/fastify-generators/src/generators/core/readme/index.ts
packages/fastify-generators/src/generators/core/readme/templates/README.md
Replaced static README generation with dynamic templating using functions from @halfdomelabs/sync and added a new README template file.
packages/project-builder-cli/src/commands/build.ts
packages/project-builder-cli/src/commands/extract-templates.ts
packages/project-builder-cli/src/index.ts
packages/project-builder-cli/src/services/schema-parser-context.ts
Introduced new CLI commands (addBuildCommand, addExtractTemplatesCommand), removed legacy generate command, and moved schema parser context logic to a dedicated file.
packages/project-builder-lib/... Removed dependency on sort-keys and replaced prettyStableStringify with stringifyPrettyStable from @halfdomelabs/utils.
packages/project-builder-server/... Added dependency on globby, updated environment flags, and modified server/runner modules to integrate template metadata extraction.
packages/sync/... Extensive updates to the template extractor framework including new extractor classes (raw and text), updated generator engine options (template metadata), new utility functions, and additional tests.
packages/project-builder-web/... Added environment variable VITE_ENABLE_TEMPLATE_EXTRACTOR, a new navigation route, and a settings page component for template extractor configuration.
packages/utils/... Added new export for JSON utilities and updated stringification functions, notably stringifyPrettyCompact and new stringifyPrettyStable.

Sequence Diagram(s)

sequenceDiagram
    participant U as User
    participant CLI as CLI Shell
    participant ETC as ExtractTemplatesCommand
    participant UT as Utility Functions
    participant SPC as SchemaParserContext
    participant RTP as runTemplateExtractorsForProject
    participant GB as Globby (File Finder)
    participant EX as Template File Extractors

    U->>CLI: Execute "pnpm --filter @halfdomelabs/project-builder-cli extract:templates"
    CLI->>ETC: Invoke addExtractTemplatesCommand()
    ETC->>UT: Call expandPathWithTilde(directory)
    UT-->>ETC: Return resolved directory
    ETC->>SPC: Create schema parser context
    SPC-->>ETC: Return context
    ETC->>RTP: Call runTemplateExtractorsForProject(directory, context)
    RTP->>GB: Search for .generator-info.json files
    GB-->>RTP: Return list of metadata files
    RTP->>EX: Process each file with appropriate extractor
    EX-->>RTP: Template extraction completed
    RTP-->>ETC: Extraction process finished
    ETC-->>CLI: Command execution complete
Loading
sequenceDiagram
    participant U as User
    participant WEB as Web App
    participant CFG as Config Service
    participant NAV as Navigation Menu
    participant SET as TemplateExtractorSettingsPage

    U->>WEB: Open settings page
    WEB->>CFG: Retrieve ENABLE_TEMPLATE_EXTRACTOR flag
    CFG-->>WEB: Return flag value
    alt Flag is true
        WEB->>NAV: Render "Template extractor" link
        U->>NAV: Click "Template extractor" link
        NAV->>SET: Load TemplateExtractorSettingsPage
    else Flag is false
        NAV-->>U: Display error alert regarding disabled template extractor
    end
Loading

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai plan to trigger planning for file edits and PR creation.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🧹 Nitpick comments (35)
packages/project-builder-server/package.json (1)

54-54: Consider using consistent version specifier format

While the addition of the globby dependency is appropriate for the new template extraction functionality, I notice that it uses a caret prefix (^14.0.2), whereas most other dependencies in this file use exact versions (no prefix). For consistency, consider either removing the caret or ensuring all dependencies follow the same versioning pattern.

-    "globby": "^14.0.2",
+    "globby": "14.0.2",
packages/project-builder-lib/src/feature-flags/flags.ts (1)

1-1: Appropriate addition of feature flag

Adding 'TEMPLATE_EXTRACTOR' to the AVAILABLE_FLAGS array establishes proper typing support for the new feature flag. This change ensures type safety when referencing this flag throughout the codebase.

Consider documenting the purpose of this flag either in a comment or in a separate documentation file to help other developers understand when and why they might want to enable this feature.

-export const AVAILABLE_FLAGS = ['TEMPLATE_EXTRACTOR'] as const;
+/**
+ * List of available feature flags:
+ * - TEMPLATE_EXTRACTOR: Enables template extraction functionality
+ */
+export const AVAILABLE_FLAGS = ['TEMPLATE_EXTRACTOR'] as const;
packages/utils/src/json/stringify-pretty-stable.ts (1)

5-10: Consider adding error handling for non-object inputs.

The function looks well-designed and provides stable JSON output through key sorting. You may want to add error handling for non-object inputs or potential exceptions from the underlying functions.

 export function stringifyPrettyStable(value: Record<string, unknown>): string {
+  if (value === null || typeof value !== 'object' || Array.isArray(value)) {
+    throw new Error('Input must be a non-null, non-array object');
+  }
   const sortedValue = sortKeys(value, {
     deep: true,
   });
   return `${stringifyPrettyCompact(sortedValue)}\n`;
 }
packages/sync/src/engine/engine.ts (1)

38-48: JSDoc comment needs to be updated to match new parameter

The JSDoc comment still references the logger parameter, but the method signature now uses an options parameter of type ExecuteGeneratorEntryOptions.

 /**
  * Builds the root generator entry.
  *
  *
  * @param rootEntry - The root generator entry.
- * @param logger - The logger to use.
+ * @param options - The options for executing the generator entry.
  */
 async build(
   rootEntry: GeneratorEntry,
   options: ExecuteGeneratorEntryOptions,
 ): Promise<GeneratorOutput> {
   return executeGeneratorEntry(rootEntry, options);
 }
packages/fastify-generators/src/generators/core/readme/templates/README.md (1)

1-27: Good basic README template with templating support

The template provides essential project information and setup instructions, using the TPL_ variable format for dynamic content.

Consider enhancing the template with:

  1. Additional placeholders for project description or repository links
  2. A section on testing strategies
  3. More detailed instructions for common development workflows
packages/sync/src/templates/text-template/render-text-template-action.ts (3)

52-59: Consider making TPL_ prefix configurable

While enforcing a prefix for template variables is a good practice, hardcoding "TPL_" may limit flexibility in the future.

+interface RenderTextTemplateOptions {
+  variablePrefix?: string;
+}

 export function renderTextTemplateFileAction<
   T extends TextTemplateFile = TextTemplateFile,
 >({
   template,
   id,
   destination,
   options,
   variables,
+  templateOptions = { variablePrefix: 'TPL_' },
 }: RenderTextTemplateActionInput<T> & { templateOptions?: RenderTextTemplateOptions }): BuilderAction {
   // ...
   
   if (variablesObj && Object.keys(variablesObj).length > 0) {
     // make sure all variables begin with TPL_
     const invalidVariableKey = Object.keys(variablesObj).find(
-      (key) => !key.startsWith('TPL_'),
+      (key) => !key.startsWith(templateOptions.variablePrefix),
     );
     if (invalidVariableKey) {
       throw new Error(
-        `Template variable must start with TPL_: ${invalidVariableKey}`,
+        `Template variable must start with ${templateOptions.variablePrefix}: ${invalidVariableKey}`,
       );
     }

81-93: Improve regex pattern for template variable replacement

The current regex pattern is limited to uppercase letters, numbers, and underscores. Consider making it more flexible while maintaining the required prefix.

 renderedTemplate = renderedTemplate.replaceAll(
-  new RegExp(/{{TPL_[A-Z0-9_]+}}/g),
+  new RegExp(`{{${templateOptions.variablePrefix}[A-Za-z0-9_]+}}`, 'g'),
   (match) => {
     const key = match.slice(2, -2);
     const value = variablesObj[key];
     if (value === undefined) {
       throw new Error(`Template variable not found: ${key}`);
     }

     return value;
   },
 );

69-79: Add comment explaining the purpose of the variable value validation

The validation preventing template variables from already existing in the template is important but may not be immediately obvious why it's necessary.

-// throw an error if variables are found in the template since this would glitch the build
+// This validation ensures that the template extractor can correctly identify variable placeholders.
+// If a variable value already exists in the template, it could lead to incorrect extraction because
+// the system wouldn't be able to distinguish between the original text and the replaced variables.
 const invalidVariableValue = Object.values(variablesObj).find(
   (val) => val !== undefined && renderedTemplate.includes(val),
 );
packages/project-builder-cli/src/commands/extract-templates.ts (1)

15-25: Command implementation follows best practices

The command is well-structured with a clear description and appropriate async action handler. The implementation follows the command pattern used elsewhere in the codebase.

Consider adding some basic validation or error handling for the directory parameter to provide more helpful error messages if an invalid directory is specified:

.action(async (directory: string) => {
+  if (!directory) {
+    logger.error('Directory parameter is required');
+    return;
+  }
  const resolvedDirectory = expandPathWithTilde(directory);
+  try {
    const context = await createSchemaParserContext(resolvedDirectory);
    await runTemplateExtractorsForProject(resolvedDirectory, context, logger);
+  } catch (error) {
+    logger.error(`Failed to extract templates: ${error.message}`);
+  }
});
packages/sync/src/templates/raw-template/raw-template-file-extractor.unit.test.ts (1)

15-56: Comprehensive test case that verifies core functionality

The test verifies that the RawTemplateFileExtractor correctly extracts and writes template files based on the provided metadata.

Consider adding additional test cases for edge scenarios:

  1. Error handling when files don't exist
  2. Behavior with empty files
  3. Handling of binary files or files with special characters
  4. Testing with nested directory structures
it('should handle non-existent files gracefully', async () => {
  const context = TemplateFileExtractorTestUtils.createTestTemplateFileExtractorContext();
  const extractor = new RawTemplateFileExtractor(context);
  
  // No files in the virtual file system
  
  // This should not throw but may log an error
  await extractor.extractTemplateFiles([
    {
      path: '/root/test-generator/missing.txt',
      metadata: {
        type: RAW_TEMPLATE_TYPE,
        generator: TemplateFileExtractorTestUtils.TEST_GENERATOR_NAME,
        template: 'missing.txt',
      },
    }
  ]);
  
  // Verify no files were created
  const result = vol.toJSON();
  expect(
    result[TemplateFileExtractorTestUtils.templatePath('missing.txt')]
  ).toBeUndefined();
});
packages/sync/src/templates/text-template/text-template-file-extractor.ts (1)

29-35: Sequential file processing could be optimized

The current implementation processes files sequentially, which may not be optimal for large numbers of files.

Consider using Promise.all to process files in parallel for better performance:

async extractTemplateFiles(
  files: TemplateFileExtractorFile<TextTemplateFileMetadata>[],
): Promise<void> {
-  for (const file of files) {
-    await this.extractTemplateFile(file);
-  }
+  await Promise.all(
+    files.map(file => this.extractTemplateFile(file))
+  );
}

However, be aware that parallel processing might lead to higher resource usage, so this optimization should be balanced against your specific requirements.

packages/utils/src/json/stringify-pretty-compact.unit.test.ts (1)

81-122: Consider adding test for maxNesting option

The function being tested supports a maxNesting option parameter, but there's no explicit test for this functionality.

You could add a test case that verifies the behavior of deeply nested objects with different maxNesting values:

+ it('should respect maxNesting option', () => {
+   const deepObj = { a: { b: { c: { d: 1 } } } };
+   const withMaxNesting = stringifyPrettyCompact(deepObj, { maxNesting: 1 });
+   const withoutMaxNesting = stringifyPrettyCompact(deepObj);
+   expect(withMaxNesting).not.toEqual(withoutMaxNesting);
+   expect(withMaxNesting.split('\n').length).toBeLessThan(withoutMaxNesting.split('\n').length);
+ });
packages/sync/src/templates/raw-template/render-raw-template-action.unit.test.ts (1)

51-69: Consider adding a test for passing custom options

The renderRawTemplateFileAction accepts an optional options parameter that gets passed to builder.writeFile(), but there's no test verifying this functionality.

You could add a test case that verifies custom options are correctly passed through:

+ it('should pass custom options to writeFile', async () => {
+   const customOptions = { encoding: 'utf-8', flag: 'w' };
+   const action = renderRawTemplateFileAction({
+     template: {
+       source: {
+         contents: Buffer.from('test content'),
+       },
+     },
+     id: 'test-id',
+     destination: 'output/test.txt',
+     options: customOptions,
+   });
+ 
+   const output = await testAction(action);
+ 
+   const file = output.files.get('output/test.txt');
+   expect(file?.options?.encoding).toBe('utf-8');
+   expect(file?.options?.flag).toBe('w');
+ });
packages/sync/src/templates/text-template/render-text-template-action.unit.test.ts (1)

161-192: Fix typo in test description

There's a small typographical error in the test description.

- it('should throw error when a variable is missing from teh template', async () => {
+ it('should throw error when a variable is missing from the template', async () => {
packages/project-builder-web/src/pages/settings/template-extractor.tsx (2)

18-76: Add documentation for component purpose and usage

The component is well-implemented but lacks documentation explaining its purpose and how it fits into the application.

Consider adding JSDoc comments to provide context:

+/**
+ * Settings page for template extractor configuration.
+ * Allows users to control template metadata generation during the extraction process.
+ * This component is rendered at the route 'settings/template-extractor'.
+ */
export function TemplateExtractorSettingsPage(): React.JSX.Element {

57-71: Enhance user guidance in the template extractor settings

The UI could provide more context about what template extraction is and why a user might want to enable/disable writing metadata.

Consider adding more explanatory text:

  <SectionList.Section>
    <SectionList.SectionHeader>
      <SectionList.SectionTitle>Settings</SectionList.SectionTitle>
+     <SectionList.SectionDescription>
+       Template extraction allows you to capture and reuse generated code as templates.
+       These settings control how the extraction process works.
+     </SectionList.SectionDescription>
    </SectionList.SectionHeader>
    <SectionList.SectionContent className="flex max-w-md flex-col gap-4">
      <CheckboxField.Controller
        name="writeMetadata"
        label="Write Metadata"
-       description="Write metadata to the project to enable template extraction"
+       description="Write metadata to the project files to enable template extraction. This allows the system to track which parts of the code were generated from templates."
        control={control}
      />
    </SectionList.SectionContent>
  </SectionList.Section>
packages/sync/src/output/builder-action-test-helpers.ts (1)

28-42: Clean implementation of the testAction function

The testAction function provides a convenient way to test individual builder actions in isolation. It handles the async flow correctly and returns the builder output after execution.

Consider adding error handling to provide more context when an action fails during testing:

export async function testAction(
  action: BuilderAction,
  context?: Partial<GeneratorTaskOutputBuilderContext>,
): Promise<GeneratorTaskOutput> {
  const builder = createTestTaskOutputBuilder(context);
-  await action.execute(builder);
+  try {
+    await action.execute(builder);
+  } catch (error) {
+    throw new Error(`Action execution failed: ${error instanceof Error ? error.message : String(error)}`);
+  }
  return builder.output;
}
packages/sync/src/templates/raw-template/raw-template-file-extractor.ts (1)

20-27: Consider parallel processing for better performance

The current implementation processes files sequentially, which may be inefficient for large numbers of files. Consider using Promise.all() to process files in parallel for better performance.

async extractTemplateFiles(
  files: TemplateFileExtractorFile<RawTemplateFileMetadata>[],
): Promise<void> {
-  for (const file of files) {
-    await this.extractTemplateFile(file);
-  }
+  await Promise.all(files.map(file => this.extractTemplateFile(file)));
}

This change would process all files concurrently rather than waiting for each file to complete before starting the next one.

packages/sync/src/templates/metadata/write-generators-metadata.ts (1)

45-76: Well-documented metadata writing function with robust implementation.

The writeGeneratorsMetadata function is well structured with:

  1. Clear JSDoc documentation explaining purpose and parameters
  2. Efficient filtering of generators with metadata
  3. Proper async/await usage
  4. Clean serialization of the generator information map

One potential improvement would be to check if generatorsWithMetadata is empty before proceeding with file writing operations, to avoid unnecessary I/O.

You could add a check like:

  const generatorsWithMetadata = new Set<string>(
    [...files.values()]
      .map((f) => f.options?.templateMetadata?.generator)
      .filter((g) => g !== undefined),
  );
+ 
+ // Skip if no generators with metadata are found
+ if (generatorsWithMetadata.size === 0) {
+   return;
+ }
packages/project-builder-cli/src/index.ts (1)

14-23: Added feature flag-based command registration for template extraction.

The code now conditionally registers the template extraction command based on the TEMPLATE_EXTRACTOR feature flag, which is a good approach for controlled feature rollout.

Consider adding a brief comment explaining the purpose of the TEMPLATE_EXTRACTOR flag:

+ // Check if template extraction feature is enabled
  if (enabledFlags.includes('TEMPLATE_EXTRACTOR')) {
    addExtractTemplatesCommand(program);
  }
packages/sync/src/templates/metadata/write-template-metadata.ts (1)

36-79: Add error handling for file writing operations

The writeTemplateMetadata function correctly uses promises for async file operations, but could benefit from more explicit error handling when writing files.

Consider adding try/catch blocks or error handling to the promises to provide better diagnostics if file writing fails:

 writePromises.push(
   fs
     .mkdir(fullDirPath, { recursive: true })
     .then(() =>
       fs.writeFile(
         metadataPath,
         JSON.stringify(sortedMetadata, null, 2),
         'utf8',
       ),
-    ),
+    )
+    .catch(error => {
+      throw new Error(`Failed to write metadata to ${metadataPath}: ${error.message}`);
+    }),
 );
packages/sync/src/templates/extractor/create-generator-info-map.ts (3)

12-12: Consider using a more specific schema definition

The generatorInfoSchema is defined as a generic record of strings, which might not catch all validation issues.

Consider defining a more specific schema that captures the expected structure of the generator info:

-const generatorInfoSchema = z.record(z.string());
+const generatorInfoSchema = z.record(
+  z.string().refine(val => val.includes('#'), {
+    message: 'Generator name must include "#" character'
+  }),
+  z.string()
+);

30-35: Improve error message formatting

The error message includes newlines with indentation that might look awkward in logs or terminals.

Consider reformatting the error message for better readability:

  if (!generatorInfoMetadata) {
    throw new Error(
-      `Could not find ${GENERATOR_INFO_FILENAME} file in ${outputDirectory}.
-         Please run a generation first with metadata writing enabled.`,
+      `Could not find ${GENERATOR_INFO_FILENAME} file in ${outputDirectory}. Please run a generation first with metadata writing enabled.`,
    );
  }

40-45: Improve error message formatting

Similar to the previous comment, the error message has formatting issues with newlines and indentation.

  if (!generatorName.includes('#')) {
    throw new Error(
-      `Generator name ${generatorName} is not in the correct format.
-             Please use the format <package-name>#<generator-name>.`,
+      `Generator name ${generatorName} is not in the correct format. Please use the format <package-name>#<generator-name>.`,
    );
  }
packages/sync/src/templates/metadata/write-generators-metadata.unit.test.ts (1)

113-173: Robust test for nested generator scenarios

This test effectively validates more complex hierarchical generator structures. The mock implementation of findNearestPackageJson is particularly well done, responding differently based on the current working directory parameter.

However, the test assertion on line 170-172 only checks for the child generator. Consider adding assertions to verify that the parent generator is handled correctly when appropriate.

 expect(metadata).toEqual({
   'child-generator': 'src/child',
+  // Consider adding assertion for parent generator if expected in output
 });
packages/project-builder-server/src/template-extractor/run-template-extractor.ts (2)

14-18: Consider making GENERATOR_PACKAGES configurable

The hard-coded array of generator packages might limit flexibility. Consider accepting an optional parameter to override or extend this list, allowing for easier extensibility in the future.

-const GENERATOR_PACKAGES = [
+export const DEFAULT_GENERATOR_PACKAGES = [
   '@halfdomelabs/core-generators',
   '@halfdomelabs/fastify-generators',
   '@halfdomelabs/react-generators',
 ];

25-47: Robust generator package mapping implementation

The function correctly handles both plugin-provided generators and predefined generators. The error handling for missing package.json files is appropriate.

One potential improvement would be adding more detailed error messages:

 if (!nearestPackageJsonPath) {
-  throw new Error(`Could not find package.json for ${packageName}`);
+  throw new Error(`Could not find package.json for ${packageName}. Searched from directory: ${path.dirname(fileURLToPath(import.meta.resolve(packageName)))}`);
 }
packages/sync/src/templates/text-template/types.ts (1)

50-59: Consider enhancing the utility function

The createTextTemplateFile function is currently a simple pass-through. Consider adding validation or default values to provide more value:

 export function createTextTemplateFile<
   T extends Record<string, TextTemplateFileVariable>,
 >(template: TextTemplateFile<T>): TextTemplateFile<T> {
+  // Add validation that type is set correctly
+  if (!template.type) {
+    template.type = TEXT_TEMPLATE_TYPE;
+  } else if (template.type !== TEXT_TEMPLATE_TYPE) {
+    throw new Error(`Invalid template type: ${template.type}. Expected: ${TEXT_TEMPLATE_TYPE}`);
+  }
   return template;
 }
packages/project-builder-server/src/sync/index.ts (2)

3-3: Avoid importing from dist directories when possible

Importing from a dist directory can lead to issues with build order and potential inconsistencies. Consider importing directly from the source module if possible.

-import type { TemplateMetadataWriterOptions } from '@halfdomelabs/sync/dist/runner/generator-runner.js';
+import type { TemplateMetadataWriterOptions } from '@halfdomelabs/sync';

164-167: Template metadata writing implementation

The conditional block for writing template metadata is well-implemented, only executing when explicitly enabled.

Consider adding a progress log to indicate that metadata writing has started/completed:

 // write metadata to the generated directory
 if (templateMetadataWriter?.enabled) {
+  logger.info('Writing template metadata...');
   await writeGeneratorsMetadata(project, output.files, projectDirectory);
   await writeTemplateMetadata(output.files, projectDirectory);
+  logger.info('Template metadata written successfully');
 }
packages/sync/src/templates/extractor/run-template-file-extractors.ts (2)

59-61: Consider memory usage for large metadata files.

Parsing all metadata files into memory with z.record(z.string(), templateFileMetadataBaseSchema.passthrough()) may lead to high memory usage if the directory is large. For massive projects, consider streaming or chunking the process.


72-79: Evaluate parallel extraction.

The extraction loop runs sequentially for each template type. If performance becomes a concern, consider parallelizing extractTemplateFiles calls, but be mindful of concurrency or I/O constraints.

packages/sync/src/templates/extractor/template-file-extractor-test-utils.ts (1)

24-25: Avoid using console directly in tests.

Using logger: console might clutter test output. Consider using a mock or custom test logger to capture logs and ensure cleaner test reporting.

packages/sync/src/templates/extractor/template-file-extractor.ts (1)

90-109: Consider alternative content comparison.

Using Buffer#equals to compare old/new contents works, but for large files, hashing or checksums might reduce memory usage. This is optional.

packages/sync/src/output/generator-task-output.ts (1)

158-158: Consider adding a test for template metadata inclusion

Make sure to add a unit test that verifies the conditional inclusion of template metadata based on the includeTemplateMetadata flag.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Lite

📥 Commits

Reviewing files that changed from the base of the PR and between 28f67c6 and 9772381.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (76)
  • .changeset/early-heads-post.md (1 hunks)
  • .cursor/rules/writing-vitest-tests.mdc (1 hunks)
  • package.json (1 hunks)
  • packages/fastify-generators/src/generators/core/readme/index.ts (2 hunks)
  • packages/fastify-generators/src/generators/core/readme/templates/README.md (1 hunks)
  • packages/project-builder-cli/package.json (1 hunks)
  • packages/project-builder-cli/src/commands/build.ts (1 hunks)
  • packages/project-builder-cli/src/commands/extract-templates.ts (1 hunks)
  • packages/project-builder-cli/src/commands/server.ts (1 hunks)
  • packages/project-builder-cli/src/index.ts (1 hunks)
  • packages/project-builder-cli/src/services/schema-parser-context.ts (1 hunks)
  • packages/project-builder-cli/tests/fixtures/server-fixture.test-helper.ts (2 hunks)
  • packages/project-builder-lib/package.json (0 hunks)
  • packages/project-builder-lib/src/definition/project-definition-container.ts (2 hunks)
  • packages/project-builder-lib/src/feature-flags/flags.ts (1 hunks)
  • packages/project-builder-lib/src/schema/index.ts (1 hunks)
  • packages/project-builder-lib/src/schema/project-definition.ts (2 hunks)
  • packages/project-builder-lib/src/schema/template-extractor/index.ts (1 hunks)
  • packages/project-builder-lib/src/utils/index.ts (0 hunks)
  • packages/project-builder-lib/src/utils/json.ts (0 hunks)
  • packages/project-builder-server/package.json (1 hunks)
  • packages/project-builder-server/src/index.ts (1 hunks)
  • packages/project-builder-server/src/runner/index.ts (4 hunks)
  • packages/project-builder-server/src/service/environment-flags.ts (1 hunks)
  • packages/project-builder-server/src/sync/index.ts (7 hunks)
  • packages/project-builder-server/src/template-extractor/index.ts (1 hunks)
  • packages/project-builder-server/src/template-extractor/run-template-extractor.ts (1 hunks)
  • packages/project-builder-web/.env.development (1 hunks)
  • packages/project-builder-web/src/pages/settings/_layout.tsx (2 hunks)
  • packages/project-builder-web/src/pages/settings/_routes.tsx (2 hunks)
  • packages/project-builder-web/src/pages/settings/template-extractor.tsx (1 hunks)
  • packages/project-builder-web/src/services/config.ts (1 hunks)
  • packages/sync/src/engine/engine.ts (2 hunks)
  • packages/sync/src/index.ts (1 hunks)
  • packages/sync/src/output/builder-action-test-helpers.ts (1 hunks)
  • packages/sync/src/output/codebase-file-reader.ts (2 hunks)
  • packages/sync/src/output/generator-task-output.ts (8 hunks)
  • packages/sync/src/output/index.ts (1 hunks)
  • packages/sync/src/runner/generator-runner.ts (5 hunks)
  • packages/sync/src/runner/generator-runner.unit.test.ts (12 hunks)
  • packages/sync/src/templates/constants.ts (1 hunks)
  • packages/sync/src/templates/extractor/create-generator-info-map.ts (1 hunks)
  • packages/sync/src/templates/extractor/create-generator-info-map.unit.test.ts (1 hunks)
  • packages/sync/src/templates/extractor/index.ts (1 hunks)
  • packages/sync/src/templates/extractor/run-template-file-extractors.ts (1 hunks)
  • packages/sync/src/templates/extractor/run-template-file-extractors.unit.test.ts (1 hunks)
  • packages/sync/src/templates/extractor/template-file-extractor-test-utils.ts (1 hunks)
  • packages/sync/src/templates/extractor/template-file-extractor.ts (1 hunks)
  • packages/sync/src/templates/index.ts (1 hunks)
  • packages/sync/src/templates/metadata/index.ts (1 hunks)
  • packages/sync/src/templates/metadata/metadata.ts (1 hunks)
  • packages/sync/src/templates/metadata/write-generators-metadata.ts (1 hunks)
  • packages/sync/src/templates/metadata/write-generators-metadata.unit.test.ts (1 hunks)
  • packages/sync/src/templates/metadata/write-template-metadata.ts (1 hunks)
  • packages/sync/src/templates/metadata/write-template-metadata.unit.test.ts (1 hunks)
  • packages/sync/src/templates/raw-template/index.ts (1 hunks)
  • packages/sync/src/templates/raw-template/raw-template-file-extractor.ts (1 hunks)
  • packages/sync/src/templates/raw-template/raw-template-file-extractor.unit.test.ts (1 hunks)
  • packages/sync/src/templates/raw-template/render-raw-template-action.ts (1 hunks)
  • packages/sync/src/templates/raw-template/render-raw-template-action.unit.test.ts (1 hunks)
  • packages/sync/src/templates/raw-template/types.ts (1 hunks)
  • packages/sync/src/templates/text-template/index.ts (1 hunks)
  • packages/sync/src/templates/text-template/render-text-template-action.ts (1 hunks)
  • packages/sync/src/templates/text-template/render-text-template-action.unit.test.ts (1 hunks)
  • packages/sync/src/templates/text-template/text-template-file-extractor.ts (1 hunks)
  • packages/sync/src/templates/text-template/text-template-file-extractor.unit.test.ts (1 hunks)
  • packages/sync/src/templates/text-template/types.ts (1 hunks)
  • packages/sync/src/templates/types.ts (1 hunks)
  • packages/sync/src/templates/utils.ts (1 hunks)
  • packages/tools/eslint-configs/typescript.js (1 hunks)
  • packages/utils/package.json (1 hunks)
  • packages/utils/src/index.ts (1 hunks)
  • packages/utils/src/json/index.ts (1 hunks)
  • packages/utils/src/json/stringify-pretty-compact.ts (3 hunks)
  • packages/utils/src/json/stringify-pretty-compact.unit.test.ts (1 hunks)
  • packages/utils/src/json/stringify-pretty-stable.ts (1 hunks)
💤 Files with no reviewable changes (3)
  • packages/project-builder-lib/package.json
  • packages/project-builder-lib/src/utils/index.ts
  • packages/project-builder-lib/src/utils/json.ts
🧰 Additional context used
🧬 Code Graph Analysis (33)
packages/project-builder-lib/src/schema/project-definition.ts (1)
packages/project-builder-lib/src/schema/template-extractor/index.ts (1)
  • templateExtractorSchema (3-10)
packages/project-builder-cli/src/services/schema-parser-context.ts (3)
packages/project-builder-lib/src/parser/types.ts (1)
  • SchemaParserContext (6-8)
packages/project-builder-common/index.js (1)
  • getDefaultPlugins (4-6)
packages/project-builder-server/src/plugins/node-plugin-store.ts (1)
  • createNodeSchemaParserContext (67-79)
packages/sync/src/engine/engine.ts (2)
packages/sync/src/runner/generator-runner.ts (1)
  • ExecuteGeneratorEntryOptions (47-50)
packages/sync/src/output/generator-task-output.ts (1)
  • GeneratorOutput (101-103)
packages/project-builder-web/src/pages/settings/_routes.tsx (1)
packages/project-builder-web/src/pages/settings/template-extractor.tsx (1)
  • TemplateExtractorSettingsPage (18-76)
packages/sync/src/output/codebase-file-reader.ts (1)
packages/utils/src/fs/handle-not-found-error.ts (1)
  • handleFileNotFoundError (8-16)
packages/project-builder-web/src/pages/settings/_layout.tsx (2)
packages/project-builder-web/src/services/config.ts (1)
  • ENABLE_TEMPLATE_EXTRACTOR (17-17)
packages/ui-components/src/components/NavigationMenu/NavigationMenu.tsx (1)
  • NavigationMenu (156-165)
packages/utils/src/json/stringify-pretty-compact.unit.test.ts (1)
packages/utils/src/json/stringify-pretty-compact.ts (1)
  • stringifyPrettyCompact (103-194)
packages/sync/src/templates/text-template/text-template-file-extractor.ts (2)
packages/sync/src/templates/text-template/types.ts (3)
  • textTemplateFileMetadataSchema (9-19)
  • TEXT_TEMPLATE_TYPE (7-7)
  • TextTemplateFileMetadata (21-23)
packages/sync/src/templates/extractor/template-file-extractor.ts (1)
  • TemplateFileExtractorFile (16-27)
packages/project-builder-web/src/pages/settings/template-extractor.tsx (5)
packages/project-builder-lib/src/schema/template-extractor/index.ts (1)
  • templateExtractorSchema (3-10)
packages/project-builder-lib/src/web/hooks/useProjectDefinition.ts (1)
  • useProjectDefinition (75-83)
packages/project-builder-lib/src/web/hooks/useResettableForm.ts (1)
  • useResettableForm (10-46)
packages/project-builder-lib/src/web/hooks/useBlockUnsavedChangesNavigate.ts (1)
  • useBlockUnsavedChangesNavigate (7-41)
packages/project-builder-web/src/services/config.ts (1)
  • ENABLE_TEMPLATE_EXTRACTOR (17-17)
packages/sync/src/templates/text-template/render-text-template-action.ts (4)
packages/sync/src/templates/text-template/types.ts (5)
  • TextTemplateFile (38-48)
  • InferTextTemplateVariablesFromTemplate (56-59)
  • TextTemplateFileVariable (28-33)
  • TextTemplateFileMetadata (21-23)
  • TEXT_TEMPLATE_TYPE (7-7)
packages/sync/src/output/generator-task-output.ts (1)
  • WriteFileOptions (22-44)
packages/sync/src/output/builder-action.ts (1)
  • BuilderAction (6-8)
packages/sync/src/templates/utils.ts (1)
  • readTemplateFileSource (12-24)
packages/sync/src/runner/generator-runner.unit.test.ts (1)
packages/sync/src/runner/generator-runner.ts (1)
  • executeGeneratorEntry (52-311)
packages/sync/src/templates/metadata/write-generators-metadata.unit.test.ts (5)
packages/sync/src/runner/tests/factories.test-helper.ts (1)
  • buildTestGeneratorEntry (52-87)
packages/sync/src/output/generator-task-output.ts (1)
  • FileData (49-62)
packages/sync/src/templates/metadata/metadata.ts (1)
  • TemplateFileMetadataBase (18-20)
packages/sync/src/templates/metadata/write-generators-metadata.ts (1)
  • writeGeneratorsMetadata (54-76)
packages/sync/src/templates/constants.ts (1)
  • GENERATOR_INFO_FILENAME (1-1)
packages/sync/src/templates/raw-template/render-raw-template-action.ts (4)
packages/sync/src/templates/raw-template/types.ts (3)
  • RawTemplateFile (24-24)
  • RawTemplateFileMetadata (17-19)
  • RAW_TEMPLATE_TYPE (7-7)
packages/sync/src/output/generator-task-output.ts (1)
  • WriteFileOptions (22-44)
packages/sync/src/output/builder-action.ts (1)
  • BuilderAction (6-8)
packages/sync/src/templates/utils.ts (1)
  • readTemplateFileSourceBuffer (32-45)
packages/sync/src/templates/raw-template/types.ts (2)
packages/sync/src/templates/metadata/metadata.ts (1)
  • templateFileMetadataBaseSchema (3-16)
packages/sync/src/templates/types.ts (1)
  • TemplateFileBase (21-26)
packages/sync/src/templates/raw-template/raw-template-file-extractor.ts (2)
packages/sync/src/templates/raw-template/types.ts (3)
  • rawTemplateFileMetadataSchema (9-12)
  • RAW_TEMPLATE_TYPE (7-7)
  • RawTemplateFileMetadata (17-19)
packages/sync/src/templates/extractor/template-file-extractor.ts (1)
  • TemplateFileExtractorFile (16-27)
packages/sync/src/templates/raw-template/render-raw-template-action.unit.test.ts (3)
packages/sync/src/templates/raw-template/render-raw-template-action.ts (1)
  • renderRawTemplateFileAction (18-49)
packages/sync/src/templates/raw-template/types.ts (2)
  • createRawTemplateFile (29-33)
  • RAW_TEMPLATE_TYPE (7-7)
packages/sync/src/output/builder-action-test-helpers.ts (1)
  • testAction (35-42)
packages/sync/src/templates/text-template/text-template-file-extractor.unit.test.ts (3)
packages/sync/src/templates/extractor/template-file-extractor-test-utils.ts (1)
  • TemplateFileExtractorTestUtils (29-41)
packages/sync/src/templates/text-template/text-template-file-extractor.ts (1)
  • TextTemplateFileExtractor (7-36)
packages/sync/src/templates/text-template/types.ts (1)
  • TEXT_TEMPLATE_TYPE (7-7)
packages/project-builder-server/src/sync/index.ts (3)
packages/sync/src/runner/generator-runner.ts (1)
  • TemplateMetadataWriterOptions (33-42)
packages/sync/src/templates/metadata/write-generators-metadata.ts (1)
  • writeGeneratorsMetadata (54-76)
packages/sync/src/templates/metadata/write-template-metadata.ts (1)
  • writeTemplateMetadata (43-79)
packages/sync/src/templates/extractor/create-generator-info-map.ts (4)
packages/sync/src/templates/extractor/template-file-extractor.ts (1)
  • TemplateFileExtractorGeneratorInfo (29-38)
packages/utils/src/fs/read-json-with-schema.ts (1)
  • readJsonWithSchema (11-31)
packages/sync/src/templates/constants.ts (1)
  • GENERATOR_INFO_FILENAME (1-1)
packages/utils/src/fs/handle-not-found-error.ts (1)
  • handleFileNotFoundError (8-16)
packages/sync/src/templates/utils.ts (1)
packages/sync/src/templates/types.ts (1)
  • TemplateFileSource (4-16)
packages/sync/src/templates/metadata/write-generators-metadata.ts (4)
packages/sync/src/generators/build-generator-entry.ts (1)
  • GeneratorEntry (53-79)
packages/utils/src/fs/find-nearest-package-json.ts (1)
  • findNearestPackageJson (24-53)
packages/sync/src/output/generator-task-output.ts (1)
  • FileData (49-62)
packages/sync/src/templates/constants.ts (1)
  • GENERATOR_INFO_FILENAME (1-1)
packages/project-builder-server/src/template-extractor/run-template-extractor.ts (4)
packages/sync/src/templates/extractor/template-file-extractor.ts (1)
  • TemplateFileExtractorCreator (132-135)
packages/project-builder-lib/src/parser/types.ts (1)
  • SchemaParserContext (6-8)
packages/sync/src/utils/evented-logger.ts (1)
  • Logger (3-8)
packages/sync/src/templates/extractor/run-template-file-extractors.ts (1)
  • runTemplateFileExtractors (26-80)
packages/sync/src/runner/generator-runner.ts (3)
packages/sync/src/utils/evented-logger.ts (1)
  • Logger (3-8)
packages/sync/src/generators/build-generator-entry.ts (1)
  • GeneratorEntry (53-79)
packages/sync/src/output/generator-task-output.ts (1)
  • GeneratorOutputMetadata (82-96)
packages/sync/src/templates/extractor/template-file-extractor.ts (3)
packages/sync/src/templates/metadata/metadata.ts (2)
  • TemplateFileMetadataBase (18-20)
  • templateFileMetadataBaseSchema (3-16)
packages/sync/src/utils/evented-logger.ts (1)
  • Logger (3-8)
packages/sync/src/templates/constants.ts (1)
  • GENERATOR_INFO_FILENAME (1-1)
packages/project-builder-lib/src/definition/project-definition-container.ts (1)
packages/utils/src/json/stringify-pretty-stable.ts (1)
  • stringifyPrettyStable (5-10)
packages/sync/src/templates/extractor/run-template-file-extractors.ts (6)
packages/sync/src/templates/extractor/template-file-extractor.ts (1)
  • TemplateFileExtractorCreator (132-135)
packages/sync/src/utils/evented-logger.ts (1)
  • Logger (3-8)
packages/sync/src/templates/extractor/create-generator-info-map.ts (1)
  • createGeneratorInfoMap (21-64)
packages/sync/src/templates/constants.ts (1)
  • TEMPLATE_METADATA_FILENAME (2-2)
packages/sync/src/templates/metadata/metadata.ts (1)
  • templateFileMetadataBaseSchema (3-16)
packages/utils/src/maps/group-by.ts (1)
  • mapGroupBy (12-27)
packages/project-builder-server/src/runner/index.ts (4)
packages/project-builder-lib/src/schema/project-definition.ts (1)
  • ProjectDefinition (63-63)
packages/utils/src/json/stringify-pretty-stable.ts (1)
  • stringifyPrettyStable (5-10)
packages/project-builder-server/src/compiler/index.ts (1)
  • compileApplications (14-44)
packages/project-builder-server/src/sync/index.ts (1)
  • generateForDirectory (106-245)
packages/sync/src/templates/metadata/write-template-metadata.ts (3)
packages/sync/src/templates/metadata/metadata.ts (1)
  • TemplateFileMetadataBase (18-20)
packages/sync/src/output/generator-task-output.ts (1)
  • FileData (49-62)
packages/sync/src/templates/constants.ts (1)
  • TEMPLATE_METADATA_FILENAME (2-2)
packages/project-builder-cli/src/commands/build.ts (3)
packages/project-builder-cli/src/utils/path.ts (1)
  • expandPathWithTilde (4-9)
packages/project-builder-cli/src/services/schema-parser-context.ts (1)
  • createSchemaParserContext (13-18)
packages/project-builder-server/src/runner/index.ts (1)
  • buildProjectForDirectory (51-75)
packages/project-builder-cli/tests/fixtures/server-fixture.test-helper.ts (1)
packages/utils/src/json/stringify-pretty-stable.ts (1)
  • stringifyPrettyStable (5-10)
packages/fastify-generators/src/generators/core/readme/index.ts (2)
packages/sync/src/templates/text-template/render-text-template-action.ts (1)
  • renderTextTemplateFileAction (30-122)
packages/sync/src/templates/text-template/types.ts (1)
  • createTextTemplateFile (50-54)
packages/sync/src/templates/extractor/run-template-file-extractors.unit.test.ts (5)
packages/sync/src/tests/logger.test-helper.ts (1)
  • createTestLogger (3-30)
packages/sync/src/templates/extractor/template-file-extractor.ts (3)
  • TemplateFileExtractorContext (40-53)
  • TemplateFileExtractorFile (16-27)
  • TemplateFileExtractorCreator (132-135)
packages/sync/src/templates/metadata/metadata.ts (1)
  • templateFileMetadataBaseSchema (3-16)
packages/sync/src/templates/constants.ts (2)
  • GENERATOR_INFO_FILENAME (1-1)
  • TEMPLATE_METADATA_FILENAME (2-2)
packages/sync/src/templates/extractor/run-template-file-extractors.ts (1)
  • runTemplateFileExtractors (26-80)
packages/sync/src/output/generator-task-output.ts (1)
packages/sync/src/templates/metadata/metadata.ts (1)
  • TemplateFileMetadataBase (18-20)
🔇 Additional comments (135)
.cursor/rules/writing-vitest-tests.mdc (1)

6-6: Helpful clarification on Vitest globals import requirement.

This addition provides important guidance for developers to include all necessary Vitest globals in their test files. This helps prevent common issues where developers might forget to import critical testing functions.

packages/project-builder-server/src/runner/index.ts (5)

7-8: Good refactoring to use standardized utilities.

The change from a custom stringifying function to the standardized stringifyPrettyStable from the utils package improves code consistency and maintainability. Adding the schema migrations import is also appropriate for the functionality being implemented.


18-18: Enhanced type safety with specific return type.

Changing the return type from Promise<unknown> to Promise<ProjectDefinition> provides better type safety and clearer contracts for function consumers.


39-39: Consistent use of new stringifyPrettyStable utility.

The new utility is correctly applied here for writing migrated definitions.


56-61: Template extraction configuration is well-implemented.

The code now correctly loads the project JSON first, then determines whether template metadata should be written based on the project configuration. This is a clean implementation that makes the template extraction feature configurable at the project level.


68-70: Template metadata writer configuration is properly passed to generator.

The template metadata writer configuration is correctly passed to the generateForDirectory function, respecting the project-level setting from the project definition.

packages/project-builder-web/.env.development (1)

4-4: Feature flag for template extractor is appropriate.

Adding the VITE_ENABLE_TEMPLATE_EXTRACTOR environment variable enables the template extraction feature in the development environment. This allows for proper testing and development of the feature.

You may want to check if this flag should also be added to other environment configurations (like production) with an appropriate default value.

packages/utils/package.json (1)

39-39:

Details

✅ Verification successful

Appropriate dependency for stable key ordering.

The addition of the sort-keys dependency is necessary for the new stringifyPrettyStable function to ensure consistent ordering of keys in JSON outputs. This is important for stable template generation and extraction.


🏁 Script executed:

#!/bin/bash
# Verify the sort-keys package usage
rg -A 2 -B 2 "sort-keys" --type ts

Length of output: 294


Dependency Verification Approved
The verification confirms that the sort-keys dependency (added at packages/utils/package.json:39) is correctly imported and used in the new stringifyPrettyStable function (in packages/utils/src/json/stringify-pretty-stable.ts). Using sort-keys to ensure consistent key ordering in JSON outputs is appropriate for stable template generation and extraction.

packages/utils/src/index.ts (1)

4-4: New JSON utility exports added

The new export makes JSON-related utilities available from the main utils package, which aligns with the existing pattern of utility exports in this file.

packages/project-builder-lib/src/schema/index.ts (1)

7-7: Template extractor schema exports added

The new export makes the template extractor functionality available through the schema index, which aligns with the PR objective of supporting template extraction and follows the existing pattern of schema exports.

packages/project-builder-server/src/template-extractor/index.ts (1)

1-2: New template extractor export structure

This new index file properly exports the template extractor functionality, following the established pattern for modular exports in the codebase.

packages/project-builder-server/src/index.ts (1)

7-7: Export of template-extractor module looks good!

This new export provides a clean way to access the template extraction functionality from outside the package.

packages/sync/src/index.ts (1)

7-7: LGTM - Export of templates module is consistent with existing pattern

The new export follows the established pattern in this file and makes the template functionality accessible to consumers of the package.

packages/utils/src/json/index.ts (1)

1-2: Good organization of JSON stringify utilities

These exports provide a clean, consolidated access point for the JSON stringify utility functions, making them easier to import and use throughout the application.

packages/sync/src/templates/constants.ts (1)

1-2: Appropriate constants for file naming

These constants follow good naming conventions (UPPER_SNAKE_CASE) and use the dot prefix convention for metadata files. Extracting these as constants will help maintain consistency if these filenames need to be referenced in multiple places.

packages/sync/src/templates/extractor/index.ts (1)

1-2: Clean exports that effectively expose template extraction functionality.

Good use of a barrel file pattern to provide a clean public API for the template extraction feature. This makes importing the functionality more straightforward for consumers.

package.json (1)

12-12: Good addition of a new script for template extraction.

This script follows the project's established pattern for delegating commands to specific packages, providing convenient access to the template extraction functionality from the project root.

packages/sync/src/output/index.ts (1)

1-1: Good export of test helper functions to support template extraction.

This addition properly exposes the builder action test helpers, making them available for use in testing the template extraction functionality. This supports comprehensive testing of the new feature.

packages/sync/src/templates/metadata/index.ts (1)

1-3: Well-organized exports for template metadata functionality.

Good use of a barrel file to aggregate metadata-related exports, providing a clean API for metadata handling. This supports the template extraction feature with proper separation of concerns.

packages/sync/src/templates/types.ts (2)

1-16: Well-designed type definition for template file sources

The TemplateFileSource union type provides a flexible approach to representing template files, allowing them to be referenced either by path or direct content. This design facilitates various use cases where templates might be read from the filesystem or generated dynamically.


18-26: Clear interface design with appropriate documentation

The TemplateFileBase interface is well-documented and establishes a good foundation for template-related entities. Using this as a base interface will allow for extensibility in derived interfaces while maintaining a consistent requirement for source information.

packages/tools/eslint-configs/typescript.js (1)

36-36: Good addition of benchmark file pattern to devDependencies

Adding support for .bench.{js,ts,jsx,tsx} files to the devDependencies list ensures that benchmark files can properly import development dependencies without triggering linting errors. This is a sensible extension that aligns with how test files are treated.

packages/project-builder-web/src/services/config.ts (2)

8-11: Consistent implementation of feature flag configuration

The implementation of VITE_ENABLE_TEMPLATE_EXTRACTOR follows the established pattern for feature flags in the codebase, using the same Zod schema approach as other flags. The transform function correctly converts string values to booleans.


17-17: Proper export of the parsed configuration value

The exported constant provides a clean way for other components to access the feature flag value without having to interact with the raw configuration object.

packages/project-builder-cli/src/services/schema-parser-context.ts (1)

1-18: Well-structured utility function for schema parser context creation

This new utility function provides a clean abstraction for creating a schema parser context, which appears to be used across multiple commands in the CLI. The function is well-documented with JSDoc comments, has proper typing, and correctly handles asynchronous operations with async/await.

The implementation properly separates concerns by delegating the actual context creation to the imported functions from other packages, making this a good example of composition.

packages/sync/src/templates/metadata/metadata.ts (1)

1-20: Well-defined schema for template file metadata

The Zod schema definition is clean and well-documented, with clear comments for each property explaining its purpose. Using Zod for runtime validation is a good practice, and inferring the TypeScript type from the schema ensures type safety and consistency.

This schema appears to be a foundational part of the new template extraction functionality, establishing a clear contract for metadata structure.

packages/project-builder-server/src/service/environment-flags.ts (1)

4-8: Environment flag update aligns with template extraction goals

The replacement of BASEPLATE_WRITE_GENERATOR_STEPS_HTML with BASEPLATE_WRITE_GENERATOR_STEPS_JSON is well-documented with a clear comment explaining its purpose. The use of double negation (!!) ensures the value is properly converted to a boolean.

This change appears to be part of the broader refactoring to support template extraction functionality across the project.

packages/project-builder-cli/src/commands/server.ts (1)

13-17: Import paths properly updated for directory structure changes

The import paths have been correctly updated to use relative paths that navigate one directory up. This change maintains the proper module resolution after what appears to be a directory restructuring.

The adjustment is consistent across all affected imports and maintains the functionality of the code.

.changeset/early-heads-post.md (1)

1-13: LGTM: Changeset file correctly documents version changes.

The file properly indicates a minor version bump for @halfdomelabs/sync and patch updates for the other packages, with a clear description of the feature being added.

packages/sync/src/templates/text-template/index.ts (1)

1-3: LGTM: Clean barrel file exports.

This barrel file provides a single entry point for template-related functionality, following best practices with explicit file extensions.

packages/project-builder-lib/src/schema/project-definition.ts (2)

16-16: LGTM: Clean import for new schema module.

The import statement follows the established pattern in this file.


58-58: LGTM: Properly adds optional templateExtractor property.

The optional templateExtractor property is added following the same pattern as other optional properties in the schema. This is a clean addition that won't break existing projects.

packages/sync/src/templates/raw-template/index.ts (1)

1-3: Well-organized barrel file that follows export patterns

This barrel file correctly exports all entities from the related modules, making them accessible through a single import point. The use of .js extensions in the imports is consistent with TypeScript's ESM module resolution.

packages/project-builder-web/src/pages/settings/_routes.tsx (2)

9-9: Import for the new template extractor settings page is correct

The import statement for the template extractor settings page follows the same pattern as other page imports in this file.


35-39: New route configuration follows existing pattern

The route configuration for the template extractor follows the same structure as the other settings routes, including the proper path, element, and breadcrumb handle.

packages/sync/src/engine/engine.ts (1)

14-14: New import for ExecuteGeneratorEntryOptions

The import of ExecuteGeneratorEntryOptions from the runner index is appropriate for the method signature update.

packages/project-builder-web/src/pages/settings/_layout.tsx (2)

6-6: Import for feature flag is correctly placed

The import for ENABLE_TEMPLATE_EXTRACTOR is appropriately added to control the visibility of the template extractor navigation item.


25-31: Feature flag implementation follows best practices

The conditional rendering of the template extractor navigation item based on the ENABLE_TEMPLATE_EXTRACTOR flag follows best practices for feature toggling. The component structure matches the pattern used for other navigation items.

packages/sync/src/templates/index.ts (1)

1-7: Well-structured barrel file for templates module.

The index.ts file provides a clean, centralized entry point for all template-related functionality. This pattern improves module organization and makes importing simpler for consumers.

packages/project-builder-cli/src/commands/build.ts (1)

14-31: Clean implementation of the build command.

The command implementation follows best practices:

  • Proper handling of the optional directory parameter with fallback to current directory
  • Appropriate path resolution with tilde expansion
  • Correct async/await patterns for context creation and project building
  • Good separation of concerns with clear dependency injection

The command structure aligns well with the project's architecture.

packages/project-builder-lib/src/schema/template-extractor/index.ts (2)

3-10: Clear schema definition with appropriate documentation.

The templateExtractorSchema is well-defined using Zod with proper JSDoc comments explaining the purpose of the writeMetadata property and its impact on template extraction.


12-14: Good type export using Zod inference.

The type definition is correctly derived from the schema using Zod's inference capabilities, ensuring type safety and consistency between the schema and TypeScript types.

packages/project-builder-cli/tests/fixtures/server-fixture.test-helper.ts (3)

9-9: Updated import for JSON stringification utility.

The import has been correctly updated to use stringifyPrettyStable from @halfdomelabs/utils package.


16-16: Updated import path for server module.

The import path for serveWebServer has been updated to reflect the new module structure.


176-183: Updated JSON stringification implementation.

The implementation has been correctly updated to use the new stringifyPrettyStable function.

packages/project-builder-lib/src/definition/project-definition-container.ts (1)

1-1: Function renamed for consistency

The change from prettyStableStringify to stringifyPrettyStable maintains the same functionality while aligning with naming conventions used throughout the codebase.

Also applies to: 107-107

packages/sync/src/output/codebase-file-reader.ts (1)

1-1: Good refactoring of error handling logic

Extracting the error handling logic into a reusable utility function improves code maintainability while preserving the same behavior.

Also applies to: 42-42

packages/sync/src/templates/text-template/render-text-template-action.ts (1)

30-122: Well-implemented template rendering with thorough validation

The renderTextTemplateFileAction function provides a robust mechanism for template rendering with good validation checks and clear error messages.

packages/utils/src/json/stringify-pretty-compact.ts (4)

66-69: Improved regex logic for compact formatting

The regex has been enhanced to better handle empty objects and arrays, allowing {} and [] to be presented in compact form. The updated regex uses positive and negative lookahead/lookbehind assertions to correctly identify isolated braces that should remain compact.


98-98: Performance improvement with replaceAll

Replacing replace with replaceAll ensures all occurrences of matched patterns are replaced in a single pass, rather than just the first occurrence. This is a good improvement for correctness and readability.


103-103: More descriptive function name

Renaming from stringify to stringifyPrettyCompact provides clearer semantic meaning and better describes the function's purpose. This improves code readability and API clarity.


113-113:

Details

✅ Verification successful

Changed default behavior for object margins

The default value for addObjectMargin has been changed from false to true, which will add spaces inside object braces by default. This makes the output more consistent with prettier formatting, but could affect existing consumers of this function.

Consider verifying that this change in default behavior doesn't break existing usages. Run tests for any code that relies on the output format to ensure consistency.


🏁 Script executed:

#!/bin/bash
# Search for direct usage of the stringifyPrettyCompact function
rg -A 3 "stringifyPrettyCompact\(" --type ts

Length of output: 5027


Default Object Margin Change Confirmed
The test output confirms that by default the function now produces spaced object braces (e.g., { "a": 1, "b": 2 }). The unit tests in stringify-pretty-compact.unit.test.ts verify both the new default behavior and the explicit use of objectMargins: false to revert to the previous formatting. Please ensure that the full test suite is run to validate no other consumers are adversely affected.

packages/project-builder-cli/src/commands/extract-templates.ts (2)

1-8: LGTM: Well-structured imports

The imports are properly organized with type imports separated from value imports, and external dependencies separated from internal imports.


9-14: Well-documented function with appropriate JSDoc

The JSDoc comment clearly explains the purpose of the function and documents the parameter.

packages/sync/src/templates/raw-template/raw-template-file-extractor.unit.test.ts (1)

1-14: Well-structured test setup with proper mocking

The test file has proper setup with vitest and memfs for in-memory file system testing. The beforeEach hook ensures a clean test environment for each test case.

packages/sync/src/templates/text-template/text-template-file-extractor.ts (1)

7-11: Well-structured class definition with clear type parameters

The class correctly extends TemplateFileExtractor with appropriate type parameters and initializes required properties.

packages/utils/src/json/stringify-pretty-compact.unit.test.ts (1)

1-123: Well-structured comprehensive test suite

This is a thorough test suite for the stringifyPrettyCompact function, covering all major functionality including simple objects, arrays, nested structures, and various formatting options.

packages/sync/src/templates/raw-template/render-raw-template-action.unit.test.ts (1)

1-70: Comprehensive test coverage for raw template rendering

The tests effectively verify both methods of providing template data (via path and direct content), ensuring template metadata is correctly applied when appropriate.

packages/sync/src/templates/text-template/render-text-template-action.unit.test.ts (3)

1-70: Good test coverage for template variable substitution

The test suite properly validates that text templates can render from both file paths and inline contents, with variables being correctly substituted.


96-127: Well-implemented error case testing

The test correctly verifies that an error is thrown when a template has a variable value pre-filled, which helps maintain the integrity of the template system.


129-159: Thorough validation of missing variables error handling

The test ensures proper error reporting when required template variables are missing, which is important for template reliability.

packages/project-builder-web/src/pages/settings/template-extractor.tsx (2)

1-17: Well-structured imports and type definition

The imports are organized clearly, and the FormData type is properly derived from the schema.


36-46: Good environmental check with informative error message

The component properly handles cases where the template extractor feature is disabled in the environment.

packages/sync/src/templates/extractor/create-generator-info-map.unit.test.ts (1)

1-76: Well-structured unit tests for createGeneratorInfoMap

The test suite is comprehensive and follows good testing practices:

  • Uses in-memory filesystem with memfs for isolation
  • Tests both success and error scenarios
  • Properly sets up and tears down test state
  • Clear assertions with meaningful error messages

The tests effectively validate that:

  1. The function can successfully map generator names to their paths
  2. It fails when the generator info file doesn't exist
  3. It fails when referenced packages aren't found in the generator package map
packages/sync/src/templates/text-template/text-template-file-extractor.unit.test.ts (1)

1-48: Effective unit test for template variable replacement

This test properly verifies that the TextTemplateFileExtractor correctly identifies and replaces variable values with template placeholders. The test:

  • Sets up a virtual file system with test content
  • Initializes the extractor with appropriate context
  • Provides metadata with variable definition (TPL_LOCATION)
  • Verifies the transformation from "Hello world from my couch!" to "Hello world from {{TPL_LOCATION}}!"
packages/sync/src/templates/raw-template/types.ts (1)

1-33: Clean implementation of raw template type definitions

The type definitions are well-structured and properly leverage Zod for schema validation. The RAW_TEMPLATE_TYPE constant and associated schema provide a foundation for template extraction without variable replacements.

The createRawTemplateFile function appears to be a simple pass-through factory function that maintains typing, which is useful for ensuring type safety when creating raw template files.

packages/sync/src/runner/generator-runner.unit.test.ts (12)

98-98: Updated function call to match new parameter structure

The executeGeneratorEntry function call has been updated to pass logger within an object, which aligns with the function's new signature that accepts options as a destructured parameter.


117-117: Updated function call to match new parameter structure

The executeGeneratorEntry function call has been updated to pass logger within an object, matching the updated function signature.


182-182: Updated function call to match new parameter structure

The executeGeneratorEntry function call has been updated to pass logger within an object parameter.


254-254: Updated function call to match new parameter structure

The executeGeneratorEntry function call has been updated to use the new parameter structure.


307-307: Updated function call to match new parameter structure

The executeGeneratorEntry function call has been updated to use the new object parameter structure.


334-334: Updated function call to match new parameter structure in error test case

The executeGeneratorEntry function call in this error test case has been updated to use the new parameter structure.


411-411: Updated function call to match new parameter structure

The executeGeneratorEntry function call has been updated to use the object parameter structure.


460-460: Updated function call to match new parameter structure in error test case

The executeGeneratorEntry function call has been updated to use the new parameter structure in this error test case.


516-516: Updated function call to match new parameter structure

The executeGeneratorEntry function call has been updated to use the object parameter structure.


556-556: Updated function call to match new parameter structure in error test case

The executeGeneratorEntry function call has been updated to use the new parameter structure in this error test case.


614-614: Updated function call to match new parameter structure in error test case

The executeGeneratorEntry function call has been updated to use the new parameter structure in this duplicate task name error test case.


679-679: Updated function call to match new parameter structure

The executeGeneratorEntry function call has been updated to use the object parameter structure in this test case for dynamic tasks with dependencies.

packages/sync/src/output/builder-action-test-helpers.ts (2)

1-8: Well-structured imports and clear type definitions

The imports are organized logically, with type imports separated from implementation imports. This makes the code more maintainable and follows TypeScript best practices.


9-26: Good implementation of the test output builder function

The createTestTaskOutputBuilder function is well-documented with JSDoc comments and provides a clean interface for testing scenarios. It correctly sets up default values for the generator configuration while allowing test-specific context overrides through the spread operator.

packages/sync/src/templates/extractor/run-template-file-extractors.unit.test.ts (4)

1-20: Well-organized imports and test setup

The test file has a good organization of imports, properly separating the test utilities from the implementation under test. The mock setup for node:fs and node:fs/promises ensures isolated tests.


21-54: Comprehensive test class setup

The test setup creates a well-structured mock extractor class that extends TemplateFileExtractor. The use of mock functions for the constructor and extract method allows for detailed verification of calls during testing.


56-136: Thorough test coverage for successful extraction scenario

This test case comprehensively verifies the file extraction process, including:

  1. Proper file system mocking with realistic template metadata
  2. Verification of context passing to extractors
  3. Validation of extractor calls with expected file data
  4. Testing of multiple file and folder structures

The detailed assertions ensure the extraction process works as expected across different file types and locations.


138-166: Good error handling test

This test properly validates the error case when no matching extractor is found for a file type. The error message assertion confirms that the implementation provides a clear, actionable error message to users.

packages/sync/src/templates/raw-template/render-raw-template-action.ts (5)

1-10: Clean import organization

The imports are well-organized, with type imports separated from implementation imports. The import of RAW_TEMPLATE_TYPE and type definitions from ./types.js follows good module organization practices.


11-16: Well-defined interface with proper type constraints

The RenderRawTemplateFileActionInput interface clearly defines the required parameters for rendering a raw template file. The use of Omit<WriteFileOptions, 'templateMetadata'> ensures that template metadata is handled consistently within the function rather than being provided by the caller.


18-30: Good implementation of the action creator and source reading

The implementation correctly creates a BuilderAction with an async execute method. It properly uses the readTemplateFileSourceBuffer utility to read the template content from the appropriate location based on the generator's base directory.


31-39: Conditional template metadata creation

The function intelligently decides whether to create template metadata based on whether the template source has a path. This allows for proper template extraction later while avoiding unnecessary metadata for inline templates.


40-49: Correct file writing with appropriate parameters

The function correctly calls builder.writeFile() with all necessary parameters, including the conditional template metadata. This ensures that files are written with the appropriate context for later extraction or processing.

packages/sync/src/templates/raw-template/raw-template-file-extractor.ts (2)

1-12: Clean class implementation with appropriate inheritance

The RawTemplateFileExtractor correctly extends the base TemplateFileExtractor class with the specific raw template metadata schema. The class properties are properly defined and initialized.


13-19: Efficient template file extraction implementation

The extractTemplateFile method follows a clean, straightforward approach by reading the source file and writing it only if modified. This prevents unnecessary file operations when content hasn't changed.

packages/fastify-generators/src/generators/core/readme/index.ts (2)

2-7: Added proper imports for template-based README generation.

The imports now include createTextTemplateFile and renderTextTemplateFileAction from the @halfdomelabs/sync package, which are necessary for implementing the templated approach to README generation.


27-47: Well-structured implementation of template-based README generation.

This change converts the README generation from a static string approach to a templated one. The implementation:

  1. Changes build to be async to accommodate the templating operations
  2. Uses renderTextTemplateFileAction to generate content from a template
  3. Creates a well-defined template with the TPL_PROJECT variable
  4. Sets appropriate formatting options

The template-based approach enhances maintainability and supports the new template extraction feature.

packages/sync/src/templates/metadata/write-generators-metadata.ts (1)

11-43: Well-implemented recursive function for building generator information.

The buildGeneratorInfoMapRecursive function efficiently builds a map of generator names to their relative paths by:

  1. Checking if the generator already exists in the map and has metadata
  2. Finding the nearest package.json for path resolution
  3. Setting paths relative to the package root
  4. Properly converting Windows backslashes to forward slashes for consistency
  5. Ensuring paths point to source directories rather than dist directories

The function also includes appropriate error handling when a package.json can't be found.

packages/project-builder-cli/src/index.ts (2)

1-1: Added shebang for direct execution of the CLI script.

Adding the Node.js shebang line allows the script to be executed directly when properly configured with executable permissions.


5-8: Updated imports to support new CLI commands.

The imports now include the necessary modules for the build command, extract templates command, serve command, and feature flag checking.

packages/project-builder-cli/package.json (3)

27-27: LGTM! Add of tsconfig improves development configuration

The addition of the --tsconfig ./tsconfig.app.json flag to the dev script ensures consistent TypeScript configuration during development.


29-29: LGTM! New template extraction script added

The new extract:templates script provides a convenient way to extract templates from generated code.


33-33: LGTM! Consistent tsconfig use in start script

Applying the same --tsconfig ./tsconfig.app.json flag to the start script ensures consistency between development and production environments.

packages/sync/src/templates/metadata/write-template-metadata.ts (1)

12-34: LGTM! Well-implemented helper function for organizing files by directory

The groupFilesByDirectory function is well-structured and effectively filters files with template metadata and groups them by directory.

packages/sync/src/templates/metadata/write-template-metadata.unit.test.ts (5)

14-22: LGTM! Well-defined interfaces for test data

The extended interfaces provide clear type definitions for the test data, enhancing type safety and readability.


37-161: LGTM! Comprehensive test for metadata file writing

This test thoroughly validates that metadata files are correctly written to the appropriate directories with the expected content.


163-211: LGTM! Good test for selective metadata writing

This test correctly verifies that the function skips files without template metadata, which is important for maintaining accurate metadata.


213-221: LGTM! Edge case testing for empty file maps

Great job testing the edge case with an empty file map, ensuring the function handles this scenario gracefully.


223-240: LGTM! Edge case testing for files without options

The test correctly verifies behavior when files don't have options, ensuring no metadata files are created in this case.

packages/sync/src/templates/extractor/create-generator-info-map.ts (1)

46-52:

Details

✅ Verification successful

Add fallback for missing package path

The function currently throws an error if a package path isn't found, but you could consider providing a fallback mechanism.

Consider verifying if this is a common scenario that might benefit from a more graceful fallback:


🏁 Script executed:

#!/bin/bash
# Check how generators are typically referenced and registered
# This helps determine if providing a fallback for missing package paths is needed

# Look for generator package registration
rg -A 5 "registerGenerator|addGenerator|generatorPackageMap" --type ts

Length of output: 12952


Generator Package Mapping Verification – Fallback Not Required

After reviewing the unit tests and the generator registration logic, it’s evident that the current behavior of throwing an error for a missing package path is intentional. The tests in create-generator-info-map.unit.test.ts explicitly expect an error (e.g., for a generator like unknown-package#core/readme), confirming that a fallback mechanism is not required. The design assumes that every generator must have a valid package path, and missing paths are considered exceptional cases.

packages/sync/src/templates/metadata/write-generators-metadata.unit.test.ts (4)

1-17: Well-structured test setup with appropriate mocking

Good use of memfs for virtual file system testing and proper mocking of Node.js filesystem modules and utility functions. The imports are organized clearly with type imports separated from function/module imports.


21-27: Good test environment preparation

The beforeEach hook appropriately resets the virtual filesystem between tests, ensuring test isolation and preventing state bleed between test cases.


29-73: Comprehensive test for files with template metadata

This test case thoroughly verifies that writeGeneratorsMetadata correctly handles files containing template metadata. The test correctly:

  1. Sets up test data with proper mocking of dependencies
  2. Verifies the output file content matches expectations
  3. Uses appropriate assertions to validate functionality

The test data structure accurately reflects real-world usage.


75-111: Proper edge case handling test

Good test coverage for the scenario where files don't contain template metadata. This edge case is important to verify that the function handles empty cases gracefully.

packages/project-builder-server/src/template-extractor/run-template-extractor.ts (1)

20-23: Well-structured extractor creators array

Good approach using a factory pattern for creating template file extractors, allowing for easy addition of new extractor types.

packages/sync/src/templates/text-template/types.ts (2)

7-19: Well-defined schema with proper validation

The textTemplateFileMetadataSchema uses Zod effectively to enforce a strict schema structure. Good use of extending the base schema to maintain consistency while adding specific fields for text templates.


21-48: Well-documented types with clear interfaces

The interfaces are well-documented with JSDoc comments that clearly explain their purpose. The generic type parameter for TextTemplateFile provides good flexibility for varying variable structures.

packages/project-builder-server/src/sync/index.ts (5)

8-9: Well-organized imports

Good addition of the necessary imports for the new template metadata functionality.


32-40: Well-documented interface extension

The new templateMetadataWriter option is properly documented with JSDoc comments explaining its purpose.


68-78: Improved error handling

The updated error handling approach is cleaner and more idiomatic. The change to use .catch(handleFileNotFoundError) directly on the promise makes the code more concise while maintaining the same functionality.


110-110: Parameter addition is consistent with interface changes

The addition of the templateMetadataWriter parameter to the function signature correctly matches the interface changes.


120-123: Proper option passing to engine build

The template metadata writer options are correctly passed to the engine build method.

packages/sync/src/templates/extractor/run-template-file-extractors.ts (1)

37-43: Handle empty or null extractor arrays.

If extractorCreators is empty, extractors would be an empty array. Confirm that the rest of the flow behaves as expected (i.e., no types in metadata or the code gracefully handles it).

Do you want to verify if the code runs gracefully with zero extractors or if we should explicitly handle this scenario?

packages/sync/src/templates/extractor/template-file-extractor.ts (3)

63-69: Confirm file-not-found handling.

Throwing an error on missing template files is valid for strict setups. If skipping missing files is desired instead, consider logging a warning and continuing.


74-78: Good validation logic.

Throwing an error when missing generator info ensures issues are caught early. No concerns here.


127-129: Extraction method design is flexible.

The abstract extractTemplateFiles() method provides a clear extension point for specialized extractors.

packages/sync/src/runner/generator-runner.ts (6)

30-42: Well-designed options interface for template metadata control

The TemplateMetadataWriterOptions interface provides a clear and well-documented way to control template metadata generation with appropriate toggles and exclusion capabilities.


44-50: Good pattern for extensible options

Using a dedicated options interface instead of individual parameters is a better approach for function extensibility, making it easier to add more options in the future without changing the function signature again.


52-54: Improved function signature using object destructuring

Changing from individual parameters to a destructured options object improves readability and maintainability.


67-67: Appropriate variable renaming

Renaming from generatorMetadatas to generatorTaskMetadatas improves clarity by making the variable name more specific and consistent with its purpose.


191-196: Clean conditional logic for template metadata inclusion

The conditional logic properly checks both if template metadata is enabled and if the current generator should be excluded, providing fine-grained control over which generators include template metadata.


294-299: Consistent variable renaming in output construction

References to the renamed variable have been consistently updated throughout the function, including in the output construction.

packages/sync/src/output/generator-task-output.ts (7)

9-9: Appropriate import for template metadata type

The import of TemplateFileMetadataBase type is correctly added to support the new template metadata functionality.


40-43: Well-documented template metadata option

The addition of the templateMetadata property to WriteFileOptions is properly documented, making it clear what this field is used for.


105-117: Well-structured interface export and enhancement

Making the GeneratorTaskOutputBuilderContext interface public with the export keyword is appropriate since it's now used as a configuration interface for external components. The new includeTemplateMetadata property is well-documented.


145-148: Clear property declaration for metadata inclusion

Adding a dedicated boolean property for metadata inclusion helps maintain clear state within the class.


158-158: Proper initialization with sensible default

The includeMetadata property is correctly initialized from the context with a default of false, which maintains backward compatibility.


187-195: Clean separation of templateMetadata from other options

Good practice to separate templateMetadata from the rest of the options in the method signature, making it clear that this is a separate concern that gets conditionally included.


210-216: Effective conditional inclusion of template metadata

The conditional logic correctly includes the templateMetadata only when includeMetadata is true, preventing unnecessary data in the output when not needed.

Comment thread packages/sync/src/templates/utils.ts
Comment thread packages/sync/src/templates/utils.ts

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

♻️ Duplicate comments (1)
packages/project-builder-server/src/template-extractor/run-template-extractor.ts (1)

49-83: Error handling implemented correctly

The implementation includes proper error handling around the runTemplateFileExtractors call as suggested in the previous review. This ensures that failures in one generator won't affect others.

🧹 Nitpick comments (6)
packages/sync/src/templates/utils.ts (1)

22-42: Buffer reading utility includes proper error handling.

The implementation correctly handles both template source types and includes error handling for file operations. The function now gracefully handles file-not-found errors with a descriptive error message, addressing concerns from previous reviews.

However, consider enhancing the error message to include the full path for better debugging:

 throw new Error(
-  `Could not find template file in project: ${source.path}`,
+  `Could not find template file in project: ${source.path} (full path: ${path.join(generatorBaseDirectory, 'templates', source.path)})`,
 );
packages/project-builder-server/src/template-extractor/run-template-extractor.ts (5)

25-47: Consider adding documentation for buildGeneratorPackageMap

The function is well-implemented, but would benefit from JSDoc comments explaining the purpose of the function and the reason for finding the nearest package.json for each generator package.

+/**
+ * Builds a map of generator package names to their directories.
+ * First adds any available plugins from the context, then finds the directory
+ * for each predefined generator package by locating its package.json.
+ * 
+ * @param context - The schema parser context containing available plugins
+ * @returns A map of generator package names to their directories
+ */
 async function buildGeneratorPackageMap(
   context: SchemaParserContext,
 ): Promise<Map<string, string>> {

36-45: Clear documentation for package resolution logic

Consider adding a comment to explain why you're locating the nearest package.json for each generator package, as this approach may not be immediately obvious to other developers.

   // attach generator packages
   for (const packageName of GENERATOR_PACKAGES) {
+    // Find the actual file location of the package by resolving it and finding its package.json
     const nearestPackageJsonPath = await findNearestPackageJson({
       cwd: path.dirname(fileURLToPath(import.meta.resolve(packageName))),
       stopAtNodeModules: true,
     });

49-53: Consider adding JSDoc for the exported function

Since this is the main exported function of the module, it would benefit from comprehensive JSDoc comments explaining its purpose, parameters, and return value.

+/**
+ * Runs template extractors for all generator info files found in the specified directory.
+ * This function discovers all `.generator-info.json` files within the project and
+ * extracts templates from the files defined in each generator.
+ *
+ * @param directory - The root directory to search for generator info files
+ * @param context - The schema parser context
+ * @param logger - Logger for output messages
+ */
 export async function runTemplateExtractorsForProject(
   directory: string,
   context: SchemaParserContext,
   logger: Logger,
 ): Promise<void> {

49-83: Consider returning extraction status

Since this function might be part of a larger workflow, consider enhancing it to return a status indicating whether all extractors ran successfully or if any errors occurred.

-export async function runTemplateExtractorsForProject(
+export async function runTemplateExtractorsForProject(
   directory: string,
   context: SchemaParserContext,
   logger: Logger,
-): Promise<void> {
+): Promise<boolean> {
+  let hasErrors = false;
   // find all .generator-info.json files in the project
   const generatorInfoFiles = await globby(
     path.join(directory, '**', '.generator-info.json'),
     { onlyFiles: true, absolute: true },
   );
   const generatorPackageMap = await buildGeneratorPackageMap(context);
   for (const generatorInfoPath of generatorInfoFiles) {
     logger.info(
       `Running template extractors for ${path.relative(
         directory,
         generatorInfoPath,
       )}...`,
     );
     const appDirectory = path.dirname(generatorInfoPath);
     await runTemplateFileExtractors(
       TEMPLATE_FILE_EXTRACTOR_CREATORS,
       appDirectory,
       generatorPackageMap,
       logger,
     ).catch((error: unknown) => {
+      hasErrors = true;
       logger.error(
         `Error running template extractors for ${path.relative(
           directory,
           generatorInfoPath,
         )}`,
       );
       logger.error(error);
     });
   }
+  return !hasErrors;
 }

55-58: Consider error handling for globby

While failures are unlikely, it's good practice to add explicit error handling for the globby call to ensure robustness.

   // find all .generator-info.json files in the project
-  const generatorInfoFiles = await globby(
-    path.join(directory, '**', '.generator-info.json'),
-    { onlyFiles: true, absolute: true },
-  );
+  let generatorInfoFiles: string[] = [];
+  try {
+    generatorInfoFiles = await globby(
+      path.join(directory, '**', '.generator-info.json'),
+      { onlyFiles: true, absolute: true },
+    );
+  } catch (error) {
+    logger.error(`Error finding generator info files: ${error}`);
+    return false;  // Return early if we can't even find the files
+  }
+
+  if (generatorInfoFiles.length === 0) {
+    logger.info('No generator info files found.');
+  }
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Lite

📥 Commits

Reviewing files that changed from the base of the PR and between 9772381 and e101806.

📒 Files selected for processing (5)
  • packages/project-builder-server/src/template-extractor/run-template-extractor.ts (1 hunks)
  • packages/sync/src/engine/engine.ts (2 hunks)
  • packages/sync/src/templates/metadata/write-template-metadata.ts (1 hunks)
  • packages/sync/src/templates/text-template/text-template-file-extractor.ts (1 hunks)
  • packages/sync/src/templates/utils.ts (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (3)
  • packages/sync/src/templates/text-template/text-template-file-extractor.ts
  • packages/sync/src/engine/engine.ts
  • packages/sync/src/templates/metadata/write-template-metadata.ts
🧰 Additional context used
🧬 Code Graph Analysis (2)
packages/project-builder-server/src/template-extractor/run-template-extractor.ts (6)
packages/sync/src/templates/extractor/template-file-extractor.ts (1)
  • TemplateFileExtractorCreator (132-135)
packages/sync/src/templates/text-template/text-template-file-extractor.ts (1)
  • TextTemplateFileExtractor (7-39)
packages/sync/src/templates/raw-template/raw-template-file-extractor.ts (1)
  • RawTemplateFileExtractor (7-27)
packages/project-builder-lib/src/parser/types.ts (1)
  • SchemaParserContext (6-8)
packages/sync/src/utils/evented-logger.ts (2)
  • Logger (3-8)
  • error (25-39)
packages/sync/src/templates/extractor/run-template-file-extractors.ts (1)
  • runTemplateFileExtractors (26-80)
packages/sync/src/templates/utils.ts (2)
packages/sync/src/templates/types.ts (1)
  • TemplateFileSource (4-16)
packages/utils/src/fs/handle-not-found-error.ts (1)
  • handleFileNotFoundError (8-16)
🔇 Additional comments (5)
packages/sync/src/templates/utils.ts (3)

1-6: Well-structured imports with clear organization.

The imports are well organized with external packages first, followed by node built-ins, and then internal types. This follows good import organization practices.


7-20: Good implementation of readTemplateFileSource with proper documentation.

The function is well-documented with JSDoc comments and efficiently implements the string reading functionality by delegating to the buffer reading function and converting the result.


44-47: Proper handling of content conversion.

The function efficiently handles the conversion between string and Buffer types when dealing with direct content sources.

packages/project-builder-server/src/template-extractor/run-template-extractor.ts (2)

1-13: Well-structured imports

The imports are organized clearly, with type imports separated from regular imports. The organization makes it easy to understand the dependencies required for this module.


14-23: Clear constant definitions

The constant arrays for generator packages and template file extractor creators are well-defined. The organization makes it straightforward to understand which generators and extractors are supported.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
packages/sync/src/templates/text-template/render-text-template-action.unit.test.ts (1)

70-94: Consider adding generatorInfo to inline template test

While testing inline template rendering, you're not explicitly passing a generatorInfo object to the testAction call, unlike other tests. This could potentially make the test less isolated if there's a default value being used.

-  const output = await testAction(action);
+  const output = await testAction(action, {
+    generatorInfo: {
+      name: 'test-generator',
+      baseDirectory: '/root/pkg/test-generator',
+    },
+  });
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Lite

📥 Commits

Reviewing files that changed from the base of the PR and between e101806 and b50d59b.

📒 Files selected for processing (5)
  • packages/project-builder-server/src/sync/index.ts (7 hunks)
  • packages/project-builder-web/src/pages/settings/template-extractor.tsx (1 hunks)
  • packages/sync/src/index.ts (1 hunks)
  • packages/sync/src/templates/text-template/render-text-template-action.ts (1 hunks)
  • packages/sync/src/templates/text-template/render-text-template-action.unit.test.ts (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (4)
  • packages/sync/src/templates/text-template/render-text-template-action.ts
  • packages/project-builder-web/src/pages/settings/template-extractor.tsx
  • packages/sync/src/index.ts
  • packages/project-builder-server/src/sync/index.ts
🧰 Additional context used
🧬 Code Graph Analysis (1)
packages/sync/src/templates/text-template/render-text-template-action.unit.test.ts (3)
packages/sync/src/templates/text-template/render-text-template-action.ts (1)
  • renderTextTemplateFileAction (30-125)
packages/sync/src/templates/text-template/types.ts (2)
  • createTextTemplateFile (50-54)
  • TEXT_TEMPLATE_TYPE (7-7)
packages/sync/src/output/builder-action-test-helpers.ts (1)
  • testAction (35-42)
🔇 Additional comments (7)
packages/sync/src/templates/text-template/render-text-template-action.unit.test.ts (7)

1-7: Well structured import organization

The imports are properly organized by external libraries first, followed by internal modules with clear separation. This follows good import organization practices.


9-14: Good test isolation practices

Properly mocking the file system modules and resetting the virtual file system before each test ensures test isolation and prevents test state pollution between test cases.


17-68: Good coverage of template path rendering with metadata

This test case thoroughly validates the core functionality of rendering a template from a file path. It checks file generation, variable replacement, and metadata preservation with comprehensive assertions.


96-127: Good validation of template extraction safety

This test correctly validates that the system prevents templates from containing values that match variable values, which could create ambiguity during template extraction. This is an important edge case to protect against.


129-159: Comprehensive error handling for missing variables

This test ensures that the system properly validates that all template variables referenced in the template must be provided in the variables object, which is crucial for correct template rendering.


161-192: Good validation of template variable usage

Validating that all declared variables are actually used in the template prevents potential confusion or errors from unused variables. This is a good practice that maintains template integrity.


1-193: Well-structured test suite with comprehensive coverage

Overall, this is a well-structured and thorough test suite. It covers:

  • The main functionality (rendering templates from both file paths and inline content)
  • Important edge cases (variable conflicts, missing variables)
  • Error conditions with specific error messages
  • Template metadata handling

The tests follow good practices for isolation, setup, and assertions.

@kingston kingston merged commit 0220038 into main Apr 10, 2025
@kingston kingston deleted the kingston/eng-498-build-reverse-generator-architecture branch April 10, 2025 11:42
@github-actions github-actions Bot mentioned this pull request Apr 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant