ESM Observability Instrumentation Guide

This guide describes a guide on enabling observability in an application which is using ECMAScript modules.

⬇️ Jump straight to Guidelines

In traditional CommonJS (CJS) module systems, developers could easily "monkey-patch" libraries during loading, allowing automatic instrumentation for performance tracking, error monitoring, and detailed tracing. The synchronous loading mechanism of require() made this straightforward.

ECMAScript Modules introduce a fundamentally different module resolution algorithm that breaks this traditional instrumentation approach. Unlike CJS, ESM:

• Loads modules asynchronously
• Builds a static module graph before evaluation
• Prevents on-the-fly modifications to imported modules

This guide will outline the current state of server-side ESM, explain the technical challenges of module instrumentation and propose solutions for observability in modern Node.js applications.

By the end of this document, developers, framework authors, and infrastructure providers will understand the nuanced landscape of ESM observability and have concrete strategies for implementation.

Context

ESM is becoming more and more a default in the server-side build output of frameworks. Here is an overview of popular Meta-Frameworks and their default build output:

Framework	Weekly Download Numbers (rounded)	Node Build Output
Next.js	8 000 000	CJS (source)
React Router	11 000 000 (v7+ framework mode probably 500k)	ESM (source)
Nuxt	700 000	ESM (source)
SvelteKit	450 000	ESM (source)
Astro	350 000	ESM (source)

Goals and Non-Goals

Goals

• Providing a guide for what's needed from a Meta-Framework to make observability instrumentation in ESM work
• Providing a guide for Cloud Infrastructure Providers to provide the necessary infrastructure for ESM-based applications

Non-Goals

• Providing a step-by-step implementation guide

Terminology

• Node Library: A library designed for use in Node.js environments providing functionality for server-side JS applications (e.g. express, mongo, graphql, postgres, ioredis)
• Observability Library: A software instrumentation tool that enables developers to track and visualize the flow of requests by generating performance traces (e.g. OpenTelemetry)
• Meta-Framework: A framework which extends a base JS framework with full-stack capabilities like server-side rendering, routing, API endpoints, and an optimized build process (e.g. Next.js, SvelteKit, Nuxt)
• Cloud Infrastructure Provider: Large-Scale cloud computing providers offering infrastructure services including storage, networking, and advanced cloud technologies (e.g. AWS, Google Cloud, Microsoft Azure)
• Deployment Platform: Specialized hosting services offering continuous deployment, content delivery and scaling for web applications (e.g. Vercel, Netlify)
• Web Application Developer: A software developer who builds web application using different libraries to create a JavaScript application

Problem Statement

As already mentioned in the introduction above, ESM is not monkey-patchable.

A short recap of the ESM module resolution algorithm: The ESM module resolution algorithm loads modules asynchronously as it is split into three phases:

1. Construction (Loading and parsing files into a module record)
2. Instantiation (linking files, creating bindings which is basically "allocating memory space")
3. Evaluation (executing code and storing actual variable values)

The import bindings are created during the 2. phase. After this, the module graph of all statically imported files is built. As the module graph is built before doing any evaluation, the static imports cannot be monkey-patched on-the-fly (like it can be done with require() in CJS).

However, module loading can be customized with customization hooks using register() and --import. In observability instrumentation libraries, loader customization and interception is often done with import-in-the-middle (e.g. in OpenTelemetry). import-in-the-middle works by wrapping modules in dummy modules that enable runtime modifications of exports, effectively working around ESM's external immutability constraints.

General Implementation Requirements

• The setup code needed for any instrumentation should run before the rest of the application (this is needed to register the Node Customization Hooks, observability instrumentation usually relies on for auto-instrumentation to "monkey-patch"). This preloading can be achieved by either:
  • Passing an instrumentation file to the Node CLI flag --import
  • The instrumentation is called and initialized at the entry point of the application, and everything else (i.e. the server code) is loaded with a dynamic import()

OR

• The Node Libraries are emitting events to the Node Diagnostics channel and the instrumentation picks this up
  • The user must use the minimum version of the Node library which already makes use of the Node Diagnostics Channel

Since the second approach is a great approach to work towards to, it requires a complex, sequential upgrade path: Node.js library authors must first implement diagnostics channel support, then observability library authors must update their tools to consume these new events, and finally, every web application developer must upgrade both their Node.js libraries and observability libraries simultaneously.

Guidelines

In the following sections, guidelines for different stakeholders are presented.

• Meta-Framework Authors
• Node Library Authors
• Cloud Infrastructure Providers and Deployment Platforms

Reference

Further Reading

• OTel: ECMAScript Modules vs CommonJS

Example Implementations

• Next.js Instrumentation Hook
  • Docs
  • PR which adds this hook
• OTel
  • PR which adds ESM support