Support prim__fork on javascript

[brainrake]

When trying to use PrimIO e => App e r on javascript, the following error is thrown:

Uncaught error: INTERNAL ERROR: No node or javascript definition found for Prelude.IO.prim__fork in ["scheme:blodwen-thread"]

I worked around it with a separate PrimIO instance which I had to then use everywhere. Implementing fork is a better solution.

In this PR, I added a javascript foreign implementation for prim__fork using setTimeout. Implementing waitThread on a timeout is not possible, but would be of arguably limited use in js (without async).

With this small change, it becomes possible to do PrimIO in App on javascript.

[mattpolzin]

Maybe this should be the browser-specific way of implementing fork. That would mean using the "browser" FFI tag instead of "javascript".

We can do better for Node than just a timeout, right? If nothing else forking via spawning a child process should be possible so I think a simple 0 second timeout for a fork in Node is a bit misleading.

[brainrake]

Thanks for the review. I've looked into it, and while setTimeout is the most widely supported implementation possible, we may be better off with something else. Javascript runs an event loop where all operations are (supposed to be) non-blocking. Forking a concurrent process in this context translates to scheduling a callback (or promise) on the event loop. This is the basic concurrency primitive provided by the runtime. To me it looks like fork is used in App as a primitive for concurrency, not parallelism, and using threads is an implementation detail.
It looks like queuing on the event loop is the fitting fork primitive on javascript and other async runtimes, to be able to use them as intended.

In fact, setTimeout works better in Nodejs because browsers add a 4ms delay on every ~5 nested setTimeouts. But it is not ideal since the delay is often long, the callback is scheduled on the next iteration of the event loop, after processing currently pending events.

Nodejs has process.nextTick and setImmediate. These are not supported on most browsers and I haven't found any advantage that would make differing runtime behavior on browser/node worthwhile.

The best API to use here would be queueMicrotask because it is fast and predictable, but not universally supported. However, Promises are almost universally supported and internally use the same mechanism.

For CPU-bound uses, web workers and node workers implement threads but have a different api and communicate via postMessage and such. They may have a place as an Idris library, but not as the main concurrency primitive. From Node API docs:

Workers (threads) are useful for performing CPU-intensive JavaScript operations. They do not help much with I/O-intensive work. The Node.js built-in asynchronous I/O operations are more efficient than Workers can be.

A mechanism for overriding FFI implementations, or interface-based reflection of backend type like in Idris1 could also be handy.

So.. I'll update the PR with an implementation that uses promises. Let me know what you think.

[brainrake]

Regarding threadWait on js: only a callback-based api can be provided (except using async, but it is not universally supported). But that can be CPS-transformed back to synchronous syntax like this: https://github.com/brainrape/idris-libuv-example/blob/master/example3.idr

[gallais]

Maybe this should be the browser-specific way of implementing fork. That would mean using the "browser" FFI tag instead of "javascript".

I thought that javascript was the browser one and node was the one for executables?

[brainrake]

according to https://idris2.readthedocs.io/en/latest/backends/javascript.html#javascript-ffi-specifiers
javascript = node + browser

[mattpolzin]

... let me know what you think.

You've now done a lot more research into this than me! I defer to you at this point. All of your notes make sense to me, though.

[gallais]

I'll update the PR with an implementation that uses promises.

Is this still the plan?

[brainrake]

Yeah. I wanted to add some tests too.

[brainrake]

I'm still on this but I ran into infinite loops while testing (trying to implement a http server and using console.log). Might be related to nodejs/node#6379 . Working towards a minimal repro or fix.

[edwinb]

This seems to still be a draft and discussion has stopped - so I'm going to close it for tidiness but please feel free to reopen if the details are rseolved.