feat: remove unused static pages (#193)

* feat: remove unused static pages

* feat: changelog path

* docs: add changelogs

Author: chinhld12

docs/adr/0007-unify-static-path-resolution.md

@@ -0,0 +1,45 @@
+# ADR: Unify Static Path Resolution for Markdown Content
+
+**Date:** 2025-05-28
+
+**Status:** Accepted
+
+**Context:**
+
+The previous system for handling static page generation involved disparate logic for resolving direct file paths, aliases, and redirects. This was spread across `getStaticPaths` and `getStaticProps` in `src/pages/[...slug].tsx`, and various helper functions in `scripts/common.ts` and `src/lib/content/paths.ts`. This led to:
+
+- Increased complexity in understanding how a URL maps to a content file.
+- Potential inconsistencies in path resolution.
+- Difficulty in maintaining and extending the system.
+- Generation of potentially unused static pages if not carefully managed.
+
+**Decision:**
+
+We decided to refactor the static path resolution mechanism to centralize and unify the logic. The core of this decision is the introduction of a new function, `getStaticJSONPaths` (located in `src/lib/content/paths.ts`).
+
+This function will be responsible for:
+
+1.  Aggregating all valid content paths from markdown files (excluding specific directories like `contributor` and `tags`).
+2.  Incorporating reversed alias mappings (where the alias target becomes the key and the alias path becomes the value).
+3.  Applying redirect rules, ensuring they don't conflict with aliases and correctly point to canonical content paths.
+4.  Providing a single, comprehensive `Record<string, string>` where keys are browser-accessible paths (e.g., `/my-alias/page`) and values are the corresponding canonical content file paths (e.g., `/actual-folder/actual-page`).
+
+The `getStaticPaths` and `getStaticProps` functions in `src/pages/[...slug].tsx` will be simplified to consume the output of `getStaticJSONPaths` directly. This makes them solely responsible for using this mapping to generate static pages and fetch content, rather than performing complex resolution logic themselves.
+
+Helper functions in `scripts/common.ts` for generating Nginx redirects and other path mappings will also be refactored to align with this unified approach, ensuring consistency between server-side redirects and application-level path resolution.
+
+**Consequences:**
+
+- **Pros:**
+  - **Simplified Logic:** Path resolution logic is now centralized, making it easier to understand, debug, and maintain.
+  - **Improved Consistency:** Ensures that all parts of the system (static generation, client-side navigation, server-side redirects) use the same source of truth for path mapping.
+  - **Reduced Redundancy:** Eliminates duplicate path resolution logic.
+  - **Leaner Builds:** By relying on a definitive list of resolvable paths, we implicitly avoid generating unused static pages.
+  - **Easier Extensibility:** Adding new types of path mappings or modifying existing ones becomes more straightforward by targeting the `getStaticJSONPaths` function.
+- **Cons:**
+  - The `getStaticJSONPaths` function becomes a critical piece of infrastructure; any bugs here will have a wide impact.
+  - Initial refactoring requires careful testing to ensure all edge cases for aliases and redirects are handled correctly.
+
+**Rationale:**
+
+The previous approach was becoming increasingly difficult to manage as the number of aliases, redirects, and content files grew. A unified system provides a more robust and scalable foundation for handling content paths. This change prioritizes maintainability and predictability in how URLs are resolved to content.

docs/changelog/0001-refactor-static-page-generation-and-paths.md

@@ -0,0 +1,11 @@
+## Refactor: Static Page Generation and Path Resolution
+
+This update introduces a significant refactoring of the system that handles static page generation, aliases, and redirects. The primary goal was to simplify logic, improve maintainability, and ensure consistent path resolution across the application.
+
+**Key Changes & Improvements:**
+
+- **Unified Path Resolution:** Introduced a new centralized function, `getStaticJSONPaths` (in `src/lib/content/paths.ts`). This function now serves as the single source of truth for mapping browser-accessible URLs to their corresponding canonical content file paths. It intelligently combines direct markdown file paths, reversed alias mappings, and filtered redirect rules.
+- **Simplified Page Generation Logic:** The `getStaticPaths` and `getStaticProps` functions within `src/pages/[...slug].tsx` have been substantially simplified. They now rely entirely on `getStaticJSONPaths` to determine valid static paths and to resolve requested URLs to the correct content files. This reduces complexity and potential inconsistencies.
+- **Removal of Unused Static Pages:** As a direct result of the more precise path resolution provided by `getStaticJSONPaths`, pages that are not explicitly defined or resolvable through this new system are no longer generated. This helps in keeping the build lean and focused on relevant content.
+- **Robust Redirect and Alias Handling:** The logic for generating Nginx redirects (`getNginxRedirects`) and other redirect mappings (`getRedirectsNotToAliases` in `scripts/common.ts`) has been refactored. This ensures that redirects are handled more robustly, especially in relation to aliases, preventing conflicts and ensuring predictable behavior.
+- **Updated Changelog URL:** The navigation link for the Changelog in the sidebar (`src/components/layout/Sidebar.tsx`) has been updated from `/updates/changelog` to `/changelog` for a cleaner URL structure.

docs/specs/0003-unified-static-path-resolution-system.md

@@ -0,0 +1,83 @@
+# Specification: Unified Static Path Resolution System
+
+**Version:** 1.0
+**Date:** 2025-05-28
+
+## 1. Introduction
+
+This document outlines the specification for a refactored system to handle static page generation, URL aliasing, and redirects. The primary objective is to create a unified, maintainable, and predictable mechanism for mapping browser-accessible URLs to their corresponding content files.
+
+## 2. Goals
+
+- Centralize path resolution logic.
+- Simplify the `getStaticPaths` and `getStaticProps` implementations in Next.js.
+- Ensure consistent handling of direct file paths, aliases, and redirects.
+- Reduce redundancy in path management code.
+- Implicitly prevent the generation of unused static pages by relying on a definitive set of resolvable paths.
+
+## 3. System Architecture
+
+The core component of the new system is the `getStaticJSONPaths` function, located in `src/lib/content/paths.ts`.
+
+### 3.1. `getStaticJSONPaths` Function
+
+- **Purpose:** To generate a comprehensive mapping of all valid browser-accessible paths to their canonical content file paths.
+- **Output:** `Promise<Record<string, string>>`
+  - Keys: Normalized browser-accessible paths (e.g., `/my-blog-post`, `/alias/to/content`). Paths will be slash-prefixed and have no trailing slash.
+  - Values: Normalized canonical content file paths relative to the content root (e.g., `/blog/my-post`, `/actual/folder/content-file`). Paths will be slash-prefixed and have no trailing slash.
+- **Logic:**
+  1.  **Fetch Raw Data:**
+      - Read `redirects.json` to get redirect rules (`getRedirectsNotToAliases` from `scripts/common.ts` will be used to pre-filter redirects that might conflict with or are superseded by aliases).
+      - Read `aliases.json` to get alias rules (`getReversedAliasPaths` from `scripts/common.ts` will be used to get a mapping of `target -> alias`).
+      - Scan the `public/content` directory to get all markdown/MDX file paths (`getAllMarkdownFiles` from `scripts/common.ts`), excluding specified directories like `contributor/` and `tags/`.
+  2.  **Path Normalization:** All paths (keys and values) will be normalized:
+      - Ensure they start with a `/`.
+      - Ensure they do not end with a `/` (unless it's the root path `/`).
+  3.  **Processing Order & Precedence:**
+      - **Markdown Files:** Direct markdown file paths form the base set of canonical content.
+      - **Aliases:** Alias paths (keys from the reversed alias map) will map to their corresponding original content paths (values from the reversed alias map). These effectively create alternative URLs for existing content.
+      - **Redirects:** Redirect source paths will map to their target paths. If a redirect target is an alias, it should resolve to the alias's canonical content path. Redirects should not overwrite existing direct markdown paths or alias paths if the source of the redirect is already a valid content path or alias.
+  4.  **Output Generation:** Combine these sources into a single record, ensuring that aliases and redirects correctly point to the final canonical content path.
+
+### 3.2. Next.js Page Generation (`src/pages/[...slug].tsx`)
+
+- **`getStaticPaths`:**
+  - Will call `getStaticJSONPaths()` once.
+  - The keys of the returned record will be transformed into the `paths` array required by Next.js.
+- **`getStaticProps`:**
+  - Will call `getStaticJSONPaths()` once.
+  - The `slug` from `params` will be used to look up the canonical content file path from the record returned by `getStaticJSONPaths`.
+  - If the requested path is not found as a key in the map, it implies a 404 (though Next.js handles this if not in `getStaticPaths`).
+  - The resolved canonical path will be used to fetch the markdown content.
+
+### 3.3. Scripting (`scripts/common.ts`)
+
+- **`getNginxRedirects`:** This function will be updated to use `getStaticJSONPaths` or a similar underlying filtered redirect list to ensure Nginx redirect rules are consistent with the application's path resolution.
+- **`getRedirectsNotToAliases`:** This utility will help in pre-filtering redirects before they are consumed by `getStaticJSONPaths` to avoid conflicts where a redirect source might also be an alias target.
+- **`getReversedAliasPaths`:** This utility provides the `target -> alias` mapping crucial for `getStaticJSONPaths`.
+
+## 4. Path Resolution Examples
+
+Assume `public/content/blog/my-article.md` exists.
+Assume `aliases.json`: `{ "/latest-post": "/blog/my-article" }`
+Assume `redirects.json`: `{ "/old-post-url": "/latest-post" }`
+
+`getStaticJSONPaths` would produce (among others):
+
+- `"/blog/my-article": "/blog/my-article"`
+- `"/latest-post": "/blog/my-article"` (from reversed alias)
+- `"/old-post-url": "/blog/my-article"` (redirect resolves through alias to canonical)
+
+## 5. UI Changes
+
+- The changelog link in `src/components/layout/Sidebar.tsx` will be updated from `/updates/changelog` to `/changelog` to reflect a cleaner URL structure, which should be a path resolvable by the new system.
+
+## 6. Non-Goals
+
+- Dynamic server-side redirects beyond what Nginx handles.
+- Client-side redirect logic (Next.js handles this based on `getStaticPaths`).
+
+## 7. Future Considerations
+
+- Performance of `getStaticJSONPaths` if the number of files, aliases, or redirects becomes extremely large.
+- More sophisticated conflict resolution strategies for paths if needed.

scripts/common.ts

@@ -1,12 +1,11 @@
 import fs from 'fs/promises';
 import path from 'path';
-import { exec } from 'child_process';
 
 const CONTENT_DIR = path.join(process.cwd(), 'public/content');
 const REDIRECTS_JSON_PATH = path.join(CONTENT_DIR, 'redirects.json');
 const ALIAS_JSON_PATH = path.join(CONTENT_DIR, 'aliases.json');
 
-function removingTrailingSlash(path: string): string {
+export function removingTrailingSlash(path: string): string {
   return path.replace(/\/$/, '');
 }
 
@@ -35,20 +34,6 @@ export async function getJSONFileContent(
   }
 }
 
-export async function execPromise(command: string): Promise<void> {
-  return new Promise((resolve, reject) => {
-    const child = exec(command, { cwd: process.cwd() }, (error) => {
-      if (error) {
-        reject(error);
-      } else {
-        resolve();
-      }
-    });
-    child.stdout?.pipe(process.stdout);
-    child.stderr?.pipe(process.stderr);
-  });
-}
-
 /**
  * Gets all markdown and MDX files recursively from a directory
  * @param dir The directory to search in
@@ -174,51 +159,127 @@ export async function getAliasPaths(): Promise<Record<string, string>> {
   return Object.assign({}, nestedAliasPaths, aliases);
 }
 
-function getIsSelfReferential(entryURL: string, targetURL: string): boolean {
-  // Check if the entry URL is a self-referential alias
-  return normalizePathWithSlash(entryURL) === normalizePathWithSlash(targetURL);
-}
-
-function getNestedRelatedAliases(
-  aliases: Record<string, string>,
-): Record<string, string> {
-  const nestedAliases: Record<string, string> = {};
-  for (const [aliasKey, aliasValue] of Object.entries(aliases)) {
-    // Split the alias key into segments
-    const segments = aliasKey.split('/').filter(Boolean);
-    // Create a new key for each segment
-    for (let i = 1; i <= segments.length; i++) {
-      const newKey = segments.slice(i).join('/');
-      if (!newKey) {
-        continue; // Skip empty keys
-      }
-      const isSelfReferential = getIsSelfReferential(newKey, aliasValue);
-      if (isSelfReferential) {
-        break;
-      }
-      if (!nestedAliases[newKey]) {
-        nestedAliases[newKey] = aliasValue;
-      }
-    }
-    // Add the original alias if it doesn't already exist
-    if (!nestedAliases[aliasKey]) {
-      nestedAliases[aliasKey] = aliasValue;
-    }
-  }
-  return nestedAliases;
-}
-
 export async function getReversedAliasPaths(): Promise<Record<string, string>> {
   const aliases = await getAliasPaths();
 
   // Reverse the keys and values
   const reversedAliases = Object.fromEntries(
     Object.entries(aliases).map(([key, value]) => [value, key]),
   );
-  return getNestedRelatedAliases(reversedAliases);
+  return reversedAliases;
 }
 
 export async function getRedirects(): Promise<Record<string, string>> {
   const redirects = await getJSONFileContent(REDIRECTS_JSON_PATH);
   return redirects;
 }
+
+async function filterRedirectsLogic(
+  processRedirect: (
+    normalizedRedirectKey: string,
+    normalizedRedirectValue: string,
+    aliasesEntries: [string, string][],
+    markdownPaths: string[],
+    filteredRedirects: Record<string, string>,
+  ) => void,
+): Promise<Record<string, string>> {
+  const redirects = await getRedirects();
+  const aliases = await getReversedAliasPaths();
+  const allMarkdownFiles = await getAllMarkdownFiles(CONTENT_DIR);
+  const aliasesEntries = Object.entries(aliases);
+
+  const markdownPaths = allMarkdownFiles
+    .filter(
+      slugArray =>
+        !slugArray[0]?.toLowerCase()?.startsWith('contributor') &&
+        !slugArray[0]?.toLowerCase()?.startsWith('tags'),
+    )
+    .map(slugArray => slugArray.join('/'));
+
+  const filteredRedirects: Record<string, string> = {};
+
+  for (const [redirectKey, redirectValue] of Object.entries(redirects)) {
+    const normalizedRedirectKey = normalizePathWithSlash(redirectKey);
+    const normalizedRedirectValue = normalizePathWithSlash(redirectValue);
+
+    const isMatchedAlias = aliasesEntries.some(([aliasKey, aliasVal]) => {
+      const normalizedAliasKey = normalizePathWithSlash(aliasKey);
+      const normalizedAliasValue = normalizePathWithSlash(aliasVal);
+      const isMatchedReversedValues =
+        normalizedRedirectValue === normalizedAliasKey ||
+        normalizedRedirectKey === normalizedAliasValue;
+      if (isMatchedReversedValues) {
+        return true;
+      }
+      return normalizedRedirectKey === normalizedAliasKey;
+    });
+
+    if (!isMatchedAlias) {
+      processRedirect(
+        normalizedRedirectKey,
+        normalizedRedirectValue,
+        aliasesEntries,
+        markdownPaths,
+        filteredRedirects,
+      );
+    }
+  }
+  return filteredRedirects;
+}
+
+export async function getNginxRedirects(): Promise<Record<string, string>> {
+  return filterRedirectsLogic(
+    (
+      normalizedRedirectKey,
+      normalizedRedirectValue,
+      aliasesEntries,
+      markdownPaths,
+      filteredRedirects,
+    ) => {
+      const aliasRedirectValue = aliasesEntries.find(([aliasKey]) => {
+        const normalizedAliasKey = normalizePathWithSlash(aliasKey);
+        return normalizedRedirectValue === normalizedAliasKey;
+      });
+
+      if (aliasRedirectValue) {
+        filteredRedirects[normalizedRedirectKey] = aliasRedirectValue[1];
+        return;
+      }
+
+      const isMatchedMdPath = markdownPaths.find(
+        mdPath => normalizePathWithSlash(mdPath) === normalizedRedirectValue,
+      );
+      if (isMatchedMdPath) {
+        filteredRedirects[normalizedRedirectKey] = isMatchedMdPath;
+      }
+    },
+  );
+}
+
+export async function getRedirectsNotToAliases(): Promise<
+  Record<string, string>
+> {
+  return filterRedirectsLogic(
+    (
+      normalizedRedirectKey,
+      normalizedRedirectValue,
+      aliasesEntries,
+      markdownPaths,
+      filteredRedirects,
+    ) => {
+      const aliasRedirectValue = aliasesEntries.find(([aliasKey]) => {
+        const normalizedAliasKey = normalizePathWithSlash(aliasKey);
+        return normalizedRedirectValue === normalizedAliasKey;
+      });
+
+      if (!aliasRedirectValue) {
+        const isMatchedMdPath = markdownPaths.find(
+          mdPath => normalizePathWithSlash(mdPath) === normalizedRedirectValue,
+        );
+        if (!isMatchedMdPath) {
+          filteredRedirects[normalizedRedirectKey] = normalizedRedirectValue;
+        }
+      }
+    },
+  );
+}

scripts/generate-nginx-redirect-map.ts

@@ -1,10 +1,11 @@
 import fs from 'fs/promises';
 import path from 'path';
 import {
-  getReversedAliasPaths,
+  getAllMarkdownFiles,
   getJSONFileContent,
+  getNginxRedirects,
+  getReversedAliasPaths,
   normalizePathWithSlash,
-  getAllMarkdownFiles,
 } from './common.js';
 
 const CONTENT_DIR = path.join(process.cwd(), 'public/content');
@@ -59,12 +60,13 @@ async function getValidShortenPaths(alias: Record<string, string> = {}) {
 
 async function generateNginxRedirectMap() {
   const alias = await getReversedAliasPaths();
+  const redirects = await getNginxRedirects();
   const shortenRedirects = await getValidShortenPaths(alias);
 
   let mapContent = 'map $request_uri $redirect_target {\n';
   mapContent += '    default 0;\n';
 
-  const paths = [alias, shortenRedirects];
+  const paths = [alias, redirects, shortenRedirects];
   // Flatten the array of objects into a single array of objects
   const flattenedPaths = paths.reduce<Record<string, string>>((acc, obj) => {
     const mapEntries = Object.entries(obj).map<Record<string, string>>(

src/components/layout/Sidebar.tsx

@@ -24,7 +24,7 @@ const navLinks = [
   },
   { title: 'Earn', url: '/earn', Icon: MemoIcons.earn },
   { title: 'Hiring', url: '/careers', Icon: MemoIcons.careers },
-  { title: 'Changelog', url: '/updates/changelog', Icon: MemoIcons.updates },
+  { title: 'Changelog', url: '/changelog', Icon: MemoIcons.updates },
   { title: 'OGIFs', url: '/updates/ogif', Icon: MemoIcons.ogif },
 ];