Endless issues of Error: Cannot find module '....\.next\server\vendor-chunks\lib\worker.js

[jdoe802]

Link to the code that reproduces this issue

https://github.com/jdoe802/pino-pretty-mongo-minimal

To Reproduce

have been working on this for a few weeks now. I was getting errors like the following when trying to implement pino transport

⨯ uncaughtException: Error: Cannot find module 'C:\Users\<redact>\.next\server\vendor-chunks\lib\worker.js'
    at Module._resolveFilename (node:internal/modules/cjs/loader:1145:15)
    at Module._load (node:internal/modules/cjs/loader:986:27)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:174:12)
    at MessagePort.<anonymous> (node:internal/main/worker_thread:186:26)
    at [nodejs.internal.kHybridDispatch] (node:internal/event_target:820:20)
    at MessagePort.<anonymous> (node:internal/per_context/messageport:23:28) {
  code: 'MODULE_NOT_FOUND',
  requireStack: []
}
Error: the worker thread exited
    at Worker.onWorkerExit (webpack-internal:///(ssr)/./node_modules/thread-stream/index.js:201:32)
    at Worker.emit (node:events:519:28)
    at [kOnExit] (node:internal/worker:315:10)
    at Worker.<computed>.onexit (node:internal/worker:229:20)
    at Worker.callbackTrampoline (node:internal/async_hooks:130:17)
 ⨯ node_modules\thread-stream\index.js (201:0) @ Worker.onWorkerExit
 ⨯ uncaughtException: Error: the worker thread exited
    at Worker.onWorkerExit (webpack-internal:///(ssr)/./node_modules/thread-stream/index.js:201:32)
    at Worker.emit (node:events:519:28)
    at [kOnExit] (node:internal/worker:315:10)
    at Worker.<computed>.onexit (node:internal/worker:229:20)
    at Worker.callbackTrampoline (node:internal/async_hooks:130:17)

this originally seemed to be fixed with the following workaround being added to the next.config file: (the worker.js file, indexes file, and wait file were all copied from /node_modules/thread-stream/ folder)

function pinoWebpackAbsolutePath(relativePath) {
    console.log("relativepath:" + relativePath + " dirname:" +  __dirname);
    console.log(path.resolve(__dirname, relativePath));
    return path.resolve(__dirname, relativePath);
}

globalThis.__bundlerPathsOverrides = {
    'thread-stream-worker': pinoWebpackAbsolutePath('./worker.js'),
    'indexes': pinoWebpackAbsolutePath('./indexes.js'),
    'wait': pinoWebpackAbsolutePath('./wait.js'),
  };

however after further inspection some logs were outputting correctly with pino transport and others were causing this message

Error: Cannot find module 'C:\Users\<redact>\.next\server\vendor-chunks\lib\worker.js'
    at Module._resolveFilename (node:internal/modules/cjs/loader:1145:15)
    at Module._load (node:internal/modules/cjs/loader:986:27)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:174:12)
    at MessagePort.<anonymous> (node:internal/main/worker_thread:186:26)
    at [nodejs.internal.kHybridDispatch] (node:internal/event_target:820:20)
    at MessagePort.<anonymous> (node:internal/per_context/messageport:23:28)
Emitted 'error' event on Worker instance at:
    at [kOnErrorMessage] (node:internal/worker:326:10)
    at [kOnMessage] (node:internal/worker:337:37)
    at MessagePort.<anonymous> (node:internal/worker:232:57)
    at [nodejs.internal.kHybridDispatch] (node:internal/event_target:820:20)
    at MessagePort.<anonymous> (node:internal/per_context/messageport:23:28)
    at MessagePort.callbackTrampoline (node:internal/async_hooks:130:17)
    at [kOnExit] (node:internal/worker:304:5)
    at Worker.<computed>.onexit (node:internal/worker:229:20)
    at Worker.callbackTrampoline (node:internal/async_hooks:130:17) {
  code: 'MODULE_NOT_FOUND',
  requireStack: []
}

The logging transport that is triggering all these issues:

import pino, {Logger} from "pino";

export const masterLogger = pino({
    level: `${process.env.NEXT_PUBLIC_PINO_LOG_LEVEL ?? "debug"}`,
    redact: ['email', 'profileName', 'password', 'address'],
    //timestamp: () => `",timestamp":"${new Date(Date.now()).toISOString()}"`,
    transport: {
        target: 'pino-mongodb',
        options: {
            uri: process.env.MONGODB_URI,
            database: 'dev',
            collection: 'log-collection',
        },
    },
});

masterLogger.info('hello');

I've tried multiple solutions (commented out in the repo) Overall it is logging in some places but not in others in my full code repo. It logs to console in all files when I don't add the transport streams. After adding the transport streams and multiple workarounds, getting this. If next could just update their files so that worker.js was properly found in the first place many issues would be avoided.

Current vs. Expected behavior

I expected the logging to work without throwing worker.js related issues

Provide environment information

Operating System:
  Platform: win32
  Arch: x64
  Version: Windows 11 Enterprise
Binaries:
  Node: 20.16.0
  npm: N/A
  Yarn: N/A
  pnpm: N/A
Relevant Packages:
  next: 14.1.3
  eslint-config-next: 14.1.3
  react: 18.2.0
  react-dom: 18.2.0
  typescript: 5.4.2
Next.js Config:
  output: N/A

Which area(s) are affected? (Select all that apply)

Output (export/standalone), Webpack

Which stage(s) are affected? (Select all that apply)

next dev (local)

Additional context

This issue is happening locally .

[magnusriga]

I spent two days around the same issue, and subsequent ones.

After a lot of trial-and-error I figured out how to make pino work in a Next.js monorepo (Turborepo), with transports, pino-pretty, and functions for messageFormat, customPrettifier, and so on (i.e. non-streamable types).

Here is the setup, with explanations: vercel/next.js#46987 (comment)

Hope it helps (the key to the worker issue you are getting was the webpack part).