Async textmate tokenization

[hediet]

Fixes #147066

Review #174364 first!

TODOs:

• Release & adopt new vscode-textmate

[zm-cttae-archive]

As downstream user, what will change in new vscode-textmate? Will it still be UMD & same interface?

If there's an issue or PR to discuss or explain it, do please point me there!

[hediet]

As downstream user, what will change in new vscode-textmate? Will it still be UMD & same interface?

Nothing changes!

[zm-cttae-archive]

Sounds great! Will unsub, also some relevant xkcd's to reward you:

xkcd: 1172 - Workflow; xkcd: 2347 - Dependency

[alexdima]

🚀 This looks really good! I left some comments. Some notes, mostly for myself, to double check before we merge this in:

• check that a text model that is not attached to an editor doesn't do background tokenization
• check that this works OK on the web
• check that vscode-textmate is not loaded if the workbench comes up without an editor
• create build and check that this works in a built product. I think it will definitely not work because we don't package textmate.worker.ts for shipping.

[hediet]

create build and check that this works in a built product. I think it will definitely not work because we don't package textmate.worker.ts for shipping.

I need help with this one. I tried to understand how "vs/editor/common/services/editorSimpleWorker.ts" is bundled, but I wasn't able to figure it out.

[alexdima]

👏