Description
Runtime operates on a very primitive structure that basically consist of
portsthat describes channels - their names and buffer sizes, they are graph nodesconnections- edges, how those graph nodes are connected (and some details about transferring the data between them)- and
effects- first of all operators, maybe something else. computational units where real magic happens
On the other hand compiler operates on a much more complicated structure that envolves entities like: packages, types, interfaces, components. Components itself is pretty complex, they consist of: header, nodes/workers (with configurations like static ports and DI), network.
The meta-programming problem
Right now I'm investigating whether it's possible to introduce some meta-programming machinery. E.g. allow user to write programs that modifies themselves or let the runtime make some modifications for e.g. auto-balancing (horizontal scaling - spawning more operators).
It turns out that it's impossible to provide a runtime API like "import package X" (see #159) or "create new instance of component Y". All updates that can happen are only updates made by using runtime-level abstractions.