Step 1: Figure out use cases?

[addaleax]

I think one of the first things we’ll want to do is to figure out what the actual use cases for (Web)Workers in Node are, so that we have a better idea of the requirements the API and implementation have to fulfill.

I guess Workers could be applied to anything that needs a fast way to share information across some kind of parallel applications, but a lot of that can already be addressed using multiple processes and standard IPC methods, so what’s going to be most interesting is to hear what one can not do fast right now without them.

[addaleax]

I can also go first: The one time I really wanted workers in Node was when building the coverage tooling for Node core itself; Parsing and serializing the JSON coverage information is a noticeable bottleneck, so being able to distribute that across multiple threads would be nice to have.

[Fishrock123]

The first thing that comes to my mind: off-I/O-thread template rendering.

[matthewp]

Code portability.

[chrisdickinson]

Seconding @Fishrock123's suggestion of off-I/O-thread template rendering — especially for server side rendering of React. Along the lines of JSON parsing, I suspect it would allow for much more efficient architectures for things like Babel & module bundlers, since this would allow authors to hand off the main-thread-blocking JS parse to a pool of threads (& possibly do so with a cheap object transfer?)

[vladholubiev]

Parsing/Generating .csv ?

For example http://papaparse.com/ does this efficiently on client-side using web workers (I guess)

[vkurchatkin]

@chrisdickinson

possibly do so with a cheap object transfer

is that a real thing or something hypothetical?

[addaleax]

@vkurchatkin We do have an actual serialization/deserialization API now in V8, that should be a pretty good start. It’s basically what Chrome uses for their WebWorkers, so that should qualify for most people’s understanding of “cheap”.

(I’d like to keep this thread on-topic for use cases; don’t be shy to open new issues here for anything. People who go watch the repo are opting into the noise. ;))

[vkurchatkin]

@addaleax fair enough. My problem with usecases is that unless a usecase is specified well enough, you could argue that anything can be solved with current multi-process model, including all examples above. For me in process workers are about efficiency and simplicity, not usecases.

[refack]

IMHO a strong use-case as @matthewp mentioned is code portability (a.k.a isomorphism). Sorry bring up cluster again, but it was homegrown, while WebWorker has a wider acceptance as a standard.

[refack]

Another use-case that comes to mind is low-priority tasks (even I/O bound). Things like Cache-Hydration, off-line batch-processing, etc.
Current implementations depend on the OS for prioritizing the main process vs the workers, WebWorker has the goal of keeping the main "thread" responsive.

[inikulin]

For parallelization of tokenization and tree construction in parse5. This can be handy for all parsers, I guess.

[kgryte]

Numeric computation. The ability to more cheaply distribute numeric computational tasks to multiple workers, as commonly found in machine learning and analysis of larger datasets (akin to MATLAB's parfor).

With the current model, need to perform wholesale copying of data between multiple processes. With shared array buffers, workers can operate on the same buffer, allowing better memory efficiency and performance.

In short, better environment support for parallel map-reduce style operations would be highly beneficial as Node.js applications become more computationally intensive.

[alexeagle]

Compilers like TypeScript and Angular would parallelize parts of the pipeline

[refack]

[question] for the parsing and heavy computation use-case. What do see as the specific benefits of Worker? multi core utilization and/or ability to keep a main thread responsive?

[matthewp]

@refack Keeping the main thread responsive. ex, for a server not blocking incoming requests.