Failing to load vscode-oniguruma wasm (and other files) since 1.80

[jhk-mjolner]

I've tried upgrading from monaco-languageclient 6.0.3 to the latest 6.4.6, and got several errors in the browser console related to not being allowed to load local resources.
I've tried different versions, and concluded that the error first appears in version 6.3.0 which uses 1.80 of monaco-vscode-api.

Error examples from browser console:
Not allowed to load local resource: file:///C:/<PATH_TO_MY_PROJECT>/node_modules/vscode/dist/service-override/vscode-oniguruma/release/onig.wasm
Not allowed to load local resource: file:///C:/<PATH_TO_MY_PROJECT>/frontend/node_modules/vscode/dist/default-extensions/theme-defaults/light_modern.json (there's several of this kind of error, about trying to load various theme .json files)
And those files rightly aren't there, so webpack can't bundle them and serve them as resources.
The error is thrown from node_modules\vscode\dist\vscode\vs\workbench\services\textMate\browser\textMateTokenizationFeatureImpl.js in async _loadVSCodeOnigurumaWASM() where it tries to load the path 'vscode-oniguruma/../onig.wasm'

I've tried looking at some source code and believe this change might be the culprit: e641ac3 as it was introduced in 1.80, remains in 1.81.7 (latest), and changes how the path to the wasm file is resolved.
Perhaps another similar change in another commit, also introduced in 1.80, might explain the same error for the theme .json file(s)?

I'm using Angular and an extra-webpack.config.js file to add webpack configuration and have this part for loading wasm, that worked fine before version 1.80:

      {
        test: /\.(mp3|wasm|ttf)$/i,
        type: 'asset/resource',
      },
      {
        test: /\.wasm$/,
        type: 'asset/inline',
      },

Is there some configuration I can use, either for monaco-languageclient, monaco-vscode-api or webpack, that can resolve this issue?
Or does it require code changes from your side?

[CGNonofr]

I see you are on windows and I've never tested it on that platform, maybe the C:/ part breaks it? (at least for the extension files part: light_modern.json for instance`)

Regarding the wasm loading issue, did you try to add an alias from vscode-oniguruma/release/onig.wasm to the absolute file path in the webpack config?

[jhk-mjolner]

I don't expect the C:/ part to be a problem, I think the problem is that the wasm is located at
C:\<PATH_TO_MY_PROEJCT>\node_modules\vscode-oniguruma\release\onig.wasm not C:/<PATH_TO_MY_PROJECT>/node_modules/vscode/dist/service-override/vscode-oniguruma/release/onig.wasm. It's the parent path relative within node_modules that has changed, I believe.
I don't know where the /vscode/dist/service-override/ comes from.
The path before the change, in the linked commit, was explicitly in node_modules. When that was removed and left to something else to resolve, I guess it gets resolved to node_modules/vscode/dist/service-override/ for some reason.

Regarding the webpack alias, I'm not sure I'm using the feature correctly, but none of these additions helped:

    alias: {
      // 'vscode-oniguruma/../onig.wasm': resolve(__dirname, './node_modules/vscode-oniguruma/release/onig.wasm'),
      // 'vscode-oniguruma/release/onig.wasm': resolve(__dirname, './node_modules/vscode-oniguruma/release/onig.wasm'),
      // 'vscode-oniguruma/../onig.wasm': './node_modules/vscode-oniguruma/release/onig.wasm',
      // 'vscode-oniguruma/release/onig.wasm': './node_modules/vscode-oniguruma/release/onig.wasm',
      // 'vscode-oniguruma/../onig.wasm': 'C:\\<PATH_TO_MY_PROJECT>\\node_modules\\vscode-oniguruma\\release\\onig.wasm',
      // 'vscode-oniguruma/release/onig.wasm': 'C:\\<PATH_TO_MY_PROJECT>\\node_modules\\vscode-oniguruma\\release\\onig.wasm',
      // 'C:\\<PATH_TO_MY_PROJECT>\\node_modules\\vscode-oniguruma\\release\\onig.wasm': 'vscode-oniguruma/../onig.wasm',
      // 'C:\\<PATH_TO_MY_PROJECT>\\node_modules\\vscode-oniguruma\\release\\onig.wasm': 'vscode-oniguruma/release/onig.wasm',
    },

[CGNonofr]

It seems that webpack fails to resolve vscode-oniguruma\release\onig.wasm and consider it as a relative path

I'm not sure how the new URL(..., import.meta.url) syntax is handled in webpack, that's the only absolute import using this syntax in the code

[jhk-mjolner]

Yeah, it seems like that's the issue.
I'm guessing the change can't be rolled back as it fixed something else?
I've tried looking at how webpack handles new URL(), but I'm not finding anything relevant for this case.

I don't know how much experience you have with webpack, but do you have any other ideas I could try?

[CGNonofr]

isn't is related to https://stackoverflow.com/questions/74038161/angular-cli-and-loading-assets-with-import-meta-url-causing-not-allowed-to-load ?

[jhk-mjolner]

Looking at the bundled code in my browser debugger source tab, I can see the line is transformed into

const _onigWasm = new URL('vscode-oniguruma/release/onig.wasm', "file:///C:/<PATH_TO_MY_PROJECT>/node_modules/vscode/dist/service-override/textmate.js").href;

So maybe the problem somehow lies with the import.meta.url pointing at the textmate.js file, and not node_modules.

[jhk-mjolner]

isn't is related to https://stackoverflow.com/questions/74038161/angular-cli-and-loading-assets-with-import-meta-url-causing-not-allowed-to-load ?

Yes!

Adding

module.exports = {
  module: {
    parser: {
      javascript: {
        url: true,
      },
    },
  },
};

and deleting the .angular folder as suggested in https://stackoverflow.com/a/75252098 seems to have resolved the issue for me.

[CGNonofr]

both issues?

[jhk-mjolner]

Yes, also the .json file loading issue. Thanks!

In the issue linked in the answer you linked, they also suggest migrating to esbuild which Angular experimentally supports now. Do you know whether using esbuild is more or less supported by monaco-languageclient and monaco-vscode-api?

[jhk-mjolner]

Taking another look at https://github.com/TypeFox/monaco-languageclient-ng-example/blob/main/custom-webpack.config.js I can also see it already includes these lines, but only mentions a 'ttf url loading issue' which I haven't had, so I haven't used those lines.
I'll suggest to them that they add that it's now necessary to load the wasm too.

[kaisalmen]

@jhk-mjolner sorry I am late to the discussion. I was only aware of the ttf issue being solved by this setting so far.
monaco-languageclient-ng-example needs to be updated. It always falls off the table. Do you want to update it to the latest version / create a PR? Do you have time for that? 😊

[CGNonofr]

I'm not sure. esbuild is used by vite in the demo of this project, we need a small hack to make it work in dev mode:

monaco-vscode-api/demo/vite.config.ts

Lines 62 to 90 in 40ba45c

	esbuildOptions: {
	plugins: [{
	name: 'import.meta.url',
	setup ({ onLoad }) {
	// Help vite that bundles/move files in dev mode without touching `import.meta.url` which breaks asset urls
	onLoad({ filter: /.*\.js/, namespace: 'file' }, async args => {
	const code = fs.readFileSync(args.path, 'utf8')
	const assetImportMetaUrlRE = /\bnew\s+URL\s*\(\s*('[^']+'|"[^"]+"|`[^`]+`)\s*,\s*import\.meta\.url\s*(?:,\s*)?\)/g
	let i = 0
	let newCode = ''
	for (let match = assetImportMetaUrlRE.exec(code); match != null; match = assetImportMetaUrlRE.exec(code)) {
	newCode += code.slice(i, match.index)
	const path = match[1].slice(1, -1)
	const resolved = await import.meta.resolve!(path, url.pathToFileURL(args.path))
	newCode += `new URL(${JSON.stringify(url.fileURLToPath(resolved))}, import.meta.url)`
	i = assetImportMetaUrlRE.lastIndex
	}
	newCode += code.slice(i)
	return { contents: newCode }
	})
	}
	}]
	}
	},

[jhk-mjolner]

I can also see that the change to new URL() syntax and the related issues are mentioned in the readme for this repo https://github.com/CodinGame/monaco-vscode-api#troubleshooting
Maybe it could be worth mentioning the issues webpack has with the syntax and how to fix it. Maybe link https://github.com/TypeFox/monaco-languageclient-ng-example/blob/main/custom-webpack.config.js as well.

[CGNonofr]

Is it related to webpack of to the angular cli though?