File content not pre-loaded for some global declarations

[tomv564]

tslib
When using the importHelpers: true compiler option and have installed tslib, the TS compiler requests content from node_modules/tslib/tslib.d.ts.
This is not recognised as a global declaration path, and it appears that ensureReferencedFiles never receives a reference to it either, so the content is never fetched.
Adding the pattern to util.ts's globalTSPatterns made tslib resolve successfully.

typeRoots and types
The default typeRoot is at least node_modules/@types, we currently limit this to the node directory? (see /node_modules\/(?:\@|%40)types\/node\/ in globalTSPatterns)

Furthermore, adding typeRoots to tsconfig's compiler options can override this default.
This means typings from other locations (eg. typeRoots: ["./mytypes", "../mysharedtypes"]) should be loaded instead.

Finally, the typings field overrides typeRoots, so when specified as types: ['node'], then the language server should not need to load content for node_modules/@types/not_the_node_package/index.d.ts

See https://www.typescriptlang.org/docs/handbook/tsconfig-json.html

Both these dependencies cannot be properly be discovered until the tsconfig.json is parsed, it might be helpful to move some content-preloading logic to ProjectConfiguration instead.

[felixfbecker]

We use a heuristic to determine the global files that are fetched, which means there are always cases that are not fetched. I wonder whether we really have to fetch all files in the workspace instead.

[tomv564]

About fetching strategy - could some form of on-demand loading work? (when typescripts asks for a file in the structure for which no content is loaded, just load the content and return that?)
Is the "This file exists but returns no content because it wasn't preloaded" scenario valid or meaningful for both remote and local file systems?

The typeRoots must be used when a developer wants to import shared typings from outside the workspace e.g. from /company/shared_typings to /company/project1. For now, that developer would be forced to copy the typings or open the whole /company folder as workspace.

[tomv564]

Outside of typeRoots, I found another example of global typings missing - in this project.
Open any spec in src/test and you should see these diagnostics:

Cannot find name 'define'.
Cannot find name 'it'.

Mocha typings are installed at node_modules/@types/mocha, but not referenced in code. As it stands, neither ensureModuleStructure nor ensureReferencedFiles would load these files.
If I add mocha's typing path to isGlobalTsFile, the compiler can eventually resolve them.

[felixfbecker]

could some form of on-demand loading work? (when typescripts asks for a file in the structure for which no content is loaded, just load the content and return that?

File fetching is async, while TS asks for contents sync. That is one of the biggest shortcomings of the TS language service API. Only thing that could be done is throw an error, catch it, add the file, recompile, try again. But if multiple attempts file that is probably way slower than fetching everything up-front.

The typeRoots must be used when a developer wants to import shared typings from outside the workspace e.g. from /company/shared_typings to /company/project1. For now, that developer would be forced to copy the typings or open the whole /company folder as workspace.

We can load everything that matches a heuristic or is referenced somehow explicitly in tsconfig or from a file.

Mocha typings are installed at node_modules/@types/mocha, but not referenced in code. As it stands, neither ensureModuleStructure nor ensureReferencedFiles would load these files.
If I add mocha's typing path to isGlobalTsFile, the compiler can eventually resolve them.

mocha makes a lot of sense to add to the global ts whitelist. But eventually these should be fetched too. Maybe we should call ensureAllFiles() in initialize, not just ensureOwnFiles().

[tomv564]

The addition of tslib and testing libs has significantly improved the editing experience when working on this project. The lack of typeRoots support is manageable for me for now, but I can open a new issue to solve it in the future!

[felixfbecker]

Would appreciate it!
I think the better fix would be to change ensureOwnFiles() in initialize to

await ensureOwnFiles();
await ensureAllFiles();

[tomv564]

I opened #289 to cover changes to the global types loading based on tsconfig.json. The other issues reported above are solved for me.