Skip to content

Conversation

@devdumpling
Copy link
Contributor

@devdumpling devdumpling commented Jan 28, 2026

Overview

(WIP)Noticed that I'm missing types for packages that have types spread across multiple entrypoints.

I wanted to address that here, as well as better explain what is happening at a high level. I will move this into the docs site later.

What is changing here

Before this only generated docs for the main types entrypoint for a given package, which works surprisingly well for most packages.

After this change it will do that for all subpath exports (max 10 until I can verify this isn't going to be crazy) that have .d.ts entrypoints given to us by the esm x-typescript-types header (which is a fancy thing they have for helping with this that I use here.

Limitations

  • This won't work for conditional exports or packages that nest types under conditions
  • no recursing for deeply nested type exports
  • arbitrary limit of 10 subpaths (stuff like lodash-es will have many more

High level overview

The "docgen" system is a pipeline that transforms npm package types into rendered HTML documentation:

API Request → Types URL Discovery → @deno/doc WASM → Node Processing → HTML Rendering → Cached
Response

In more detail:

  1. API Route (server/api/registry/docs/[...pkg].get.ts:7-70)
  • Receives request like /api/registry/docs/nuxt@3.15.4
  • Validates package name, resolves version via npm registry
  • Calls generateDocsWithDeno(packageName, version)
  • Wraps result in 1-hour SWR cache
  1. Generation Orchestrator (server/utils/docs/index.ts:34-55)
  • Calls getDocNodes() to get raw TS AST nodes
  • Processes nodes: flatten namespaces → merge overloads → build symbol lookup
  • Renders to HTML and TOC
  • Returns null if no nodes (package has no types)
  1. Types URL Discovery (server/utils/docs/client.ts:67-138)
  • Main entry: HEAD request to esm.sh/{pkg}@{version} -> reads x-typescript-types header
  • Subpaths: Fetches package.json from npm registry → scans exports field for subpaths with types
    → queries esm.sh for each subpath's types URL
  • Returns deduplicated list of .d.ts URLs
  1. @deno/doc WASM (server/utils/docs/client.ts:42-45)
  • The doc() function from @deno/doc parses TypeScript declaration files
  • Uses custom createLoader() to fetch modules via HTTP
  • Uses custom createResolver() to resolve imports (relative paths, bare specifiers via esm.sh)
  1. Module Loader (server/utils/docs/client.ts:157-212)
  • Fetches URLs via fetch() with 30-second timeout
  • Returns undefined for non-http/https protocols (like node:*)
  • Returns undefined on fetch failures (silent)
  1. Node Processing (server/utils/docs/processing.ts)
  • flattenNamespaces(): Inlines namespace members (e.g., Utils.foo())
  • mergeOverloads(): Groups function overloads, picks best JSDoc
  • buildSymbolLookup(): Creates map for cross-reference links
  1. HTML Rendering (server/utils/docs/render.ts)
  • Groups symbols by kind (functions, classes, interfaces, etc.)
  • Renders signatures with shiki syntax highlighting
  • Processes JSDoc tags (params, returns, examples, deprecated)
  • Generates TOC with section counts

@vercel
Copy link

vercel bot commented Jan 28, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Review Updated (UTC)
docs.npmx.dev Ready Ready Preview, Comment Jan 28, 2026 3:44pm
npmx.dev Ready Ready Preview, Comment Jan 28, 2026 3:44pm

Request Review

@devdumpling devdumpling changed the title fix(docgen): various improvements fix: (docgen) various improvements Jan 28, 2026
@devdumpling devdumpling changed the title fix: (docgen) various improvements fix: (docgen) remove redundant requests, get types from more than just main entrypoint, fail gracefully-er Jan 28, 2026
@devdumpling
Copy link
Contributor Author

well this works but by gawd is it slow for some repos with many subpath exports with types (like took maybe 20s for nuxt)

@devdumpling devdumpling changed the title fix: (docgen) remove redundant requests, get types from more than just main entrypoint, fail gracefully-er fix: (docgen) get types from more than just main entrypoint, fail gracefully-er Jan 28, 2026
/**
* Extract types URLs from pre-fetched package exports.
*/
async function getSubpathTypesUrlsFromExports(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bottleneck. Going to think if there's a better way.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or rather, it's a bottleneck that we have to run deno doc potentially on a bunch of subpath exports.

}

const versionData = packument.versions?.[version]
const exports = versionData?.exports as Record<string, unknown> | undefined
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remember exports can also be a string (one entrypoint so exports: "./foo.js" is like exports: {".": "./foo.js"} iirc)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants