New theme matching and new tokens format

[alexdima]

Adopt new theme matching support available in vscode-textmate.

• merge PR with the actual work: New tokens format & theme matching  #17933

DONE:

• Common
  • adopt new tokens representation in model & co.
  • remove paranoia check in ViewLineToken
  • remove model.getMode() and model.getModeId()
  • rename LanguageIdentifier.sid and LanguageIdentifier.iid
  • remove token class name from every token
• Standalone editor
  • generate new tokens format in monarch and manual tokenizers
  • align standalone languages tokenizers to generate classic style tokens
  • clean up standalone tokens css rules and move them to JS
  • match tokens using theme rules in JS for monarch and manual tokenizers
  • expose new tokenization API in standalone editor
  • adopt standalone languages tests for tokenization
  • validate new tokenization using tokenization.html standalone test
  • allow for theme switching in standaloneColorService
  • allow for theme customization in standalone editor
  • check editor with bootstrap css (created follow up task)
  • add inspect tokens command
• adopt new vscode-textmate
  • publish vscode-textmate
  • get theme information to vscode-textmate
  • reimplement theme integration tests
  • allow for theme switching
  • handle encountered languages while tokenizing
  • remove dom layer breaker from TMSyntax.ts
  • add inspect tokens command

[alexdima]

Test plan item: #18318