Multiple environments support

[inikulin]

Problem 1: Using parse5 in browsers, ReactNative, etc.

According to recent reports (#225, #220, #234, #226, #202, #213) people try to use parse5 in various environments that are distinct from Node.js. parse5 core functionality doesn't use any platform-specific API, however some advanced features like streaming support depend on Node.js API which renders parse5 unusable in other environments.

Problem 2: TypeScript typings

The other problem is the immaturity of TypeScript type dependencies distribution system, starting from vague semi-official recommendations from development team of how such dependencies should be managed and ending with the mutually exclusive issues that this suggested approach causes: #212, #232.

The Big Plan

• Transform parse5 into set of packages. Core parsing functionality will remain in parse5 package. Any advanced functionality will be distributed in separate packages (e.g. parse5-sax, parse5-parse-stream) with a peer dependency on core package.
• Each package will have it's own TypeScript typings.
• Move Node.js typings dependency to devDependencies and document that in order to use parse5 typing one need to install Node.js typings.
• Raise issue regarding types distribution in TypeScript repo

Ad-hoc solution for now

Unfortunately, there are several blockers that prevent us from implementing The Big Plan at the moment:

• It will obviously require some significant time to implement: we'll need to adopt something like lerna to manage multiple packages. Documentation and build system will require significant rework.
• There is a huge pending change set in https://github.com/HTMLParseErrorWG/parse5 fork which we'll need to merge once the job will be done there (hopefully in the near future). Going multi-package path now will make it almost impossible to merge this change set automatically.

So suggested solution for now is:

• Make API that depends on Node.js API lazy-loadable
• Remove Node.js typings dependency and document that it should be installed manually
• Remove types field from package.json, but still distribute typings within the package but suggest approach recommended by @fictitious in Move @types/node from runtime deps to dev deps #226 (comment) to enable them.

If everyone involved in aforementioned conversations is fine with the proposed solutions then I can roll out ad-hoc solution somewhere on weekend.

Milestones

• Merge recent changes from https://github.com/HTMLParseErrorWG/parse5. Close issues fixed in that fork, update docs about breaking changes accordingly.
• Update repo tooling and codebase:
  • Get rid of gulp in favour of npm commands
  • Use prettier
  • Switch to ES6 syntax
  • Update file naming in repo to use - instead of -.
• Move streaming APIs and htmlparser2 tree-adapter to separate packages
• Adopt https://github.com/lerna/lerna for publishing
• Refactor tests and move them to separate packages
• Move TS typings to @types repo
• Update docs with recent changes
  • Use treeAdapter to attach location information #189
  • Source code location: startCol, endCol, startLine, endLine
  • Implement scriptingEnabled flag #192
  • Use token objects as SAXParser event arguments #247
  • Changelog
• Decide what to do with the documentation

[domenic]

I am happy that jsdom consumers won't get these typing dependencies installed by default. I'd be even happier if they got no types at all, just JavaScript, but I guess that would require using DefinitelyTyped, which for some reason some people believe to be "legacy".

[fictitious]

Just a wild side note about TypeScript typings. They are distributed by nature - there is no central authority that recompiles everything on every submission and enforces consistency. And until no one tried to use parse5 and ReactNative together in one project, the system was effectively partitioned.

According to CAP theorem, in the presence of a network partition, one has to choose between consistency and availability.

But with typings, inconsistency makes them completely unusable for everyone because it stops the builds, making system effectively unavailable.

The outtakes from this for those who use typings, 🤔💭 I think are

• apparent success of TypeScript is a big paradox, partially fueled by illusion of consistency it provides
• typings distribution will always be unsolved problem (partition-intolerant, or "immature", if you like)
• one has always to be prepared to deploy local hacks to unblock the builds

[inikulin]

Hi everyone.

I've just released v4.0.0 that implements proposed ad-hoc solution to npm. Now you should be able to bundle parse5 for platforms other than Node.js. Please let me know if you encounter any additional problems.

[felixfbecker]

Is there a reason why the Node typings dependency can't simply use exactly the range of Node versions this module supports?