Computed metadata fields

[rufuspollock]

Computed fields (or just any operations on incoming records)

cf https://www.contentlayer.dev/docs/reference/source-files/define-document-type#computedfields

When loading a file i want to create new fields based on some computation so that i can have additional metadata

• Add a type based on the folder of the file so that i can label blog posts
• Add layout based on the folder so i can change layouts based on folder
• compute title frontmatter from first heading

More Examples:

• Generate keywords based on the content of the file.
• Label blog posts based on the folder structure.
• Count the number of words in the content.
• Estimate the reading time based on the content.
• Detect the language of the content.

Acceptance

• Design sketch with API/UX for users of lib in README or similar
• Simple example in examples folder just a single js file like index.js

import indexFolder from markdowndb

files = indexFolder(../../__mock__, computedFields=...)

console.log(files[0].readingTime)

Tasks

• sketch out the design before implementing
• ...

Design

We can achieve this by allowing users to define custom fields using JavaScript functions. Each function takes the following information as parameters:

• File path
• File metadata (e.g., tags, title, ...)
• File body content
• File type (e.g., blog, ...)

The custom field generated by the function can then be added to the document scheme.

Example Implementation

// User-defined function to generate keywords based on content
function generateKeywords(filePath, metadata, content, fileType) {
  const keywords = ....
  return { keywords };
}

// Document scheme with custom field added
const documentScheme = {
  keywords: { value: generateKeywords}, // this will be excuted
};

In this example, the generateKeywords function is a placeholder for the user-defined function. Users can implement similar functions to automate the generation of custom fields based on their specific requirements. The custom field is then added to the document scheme for validation and type decleration.

Notes

[olayway]

@rufuspollock I've advised @mohamedsalem401 to sketch out architecture design before implementing

[mohamedsalem401]

What do we want by implementing custom fields?

We aim to provide users with a simple way to automate field generation based on a file's data.

Examples:

• Generate keywords based on the content of the file.
• Label blog posts based on the folder structure.
• Count the number of words in the content.
• Estimate the reading time based on the content.
• Detect the language of the content.

How can we do this?

We can achieve this by allowing users to define custom fields using JavaScript functions. Each function takes the following information as parameters:

• File path
• File metadata (e.g., tags, title, ...)
• File body content
• File type (e.g., blog, ...)

The custom field generated by the function can then be added to the document scheme.

Example Implementation:

// User-defined function to generate keywords based on content
function generateKeywords(filePath, metadata, content, fileType) {
  const keywords = ....
  return { keywords };
}

// Document scheme with custom field added
const documentScheme = {
  keywords: { value: generateKeywords}, // this will be excuted
};

In this example, the generateKeywords function is a placeholder for the user-defined function. Users can implement similar functions to automate the generation of custom fields based on their specific requirements. The custom field is then added to the document scheme for validation and type decleration.

[rufuspollock]

@rufuspollock i've posted a design approach in the description. let me know what you think.

[rufuspollock]

@mohamedsalem401 i think we probably want to operate on the ast ... (things are already parsed ...). at least in first pass i would try that ...

[rufuspollock]

@mohamedsalem401 can you draft your spec as a PR to README.md (for a new section called "Computed Fields"

I think the function is something like:

function computeSomething(fileObject, ast) { // either returns null and changes in place OR returns a duplicate of the object with the altered fields ...
// my guess is to KISS and just alter in place for now ...
}