Gantz Design

[mitchmindtree]

Nodes

Gantz allows for constructing executable directed graphs by composing together Nodes.

Nodes are a way to allow users to abstract and encapsulate logic into smaller, re-usable components, similar to a function in a coded programming language.

Every Node is made up of the following:

• Any number of inputs, where each input is of some rust type or generic type.
• Any number of outputs, where each output is of some rust type or generic type.
• A function that takes the inputs as arguments and returns an Outputs struct containing a field for each of the outputs.

Graphs

Graphs describe the composition of one or more nodes.

Composition

Nodes can be composed together by connecting outputs to inputs of other nodes. Connections can only be made between outputs and inputs that share the same type.

Evaluation

A graph can be evaluated in one of two ways:

1. Push evaluation. The graph allows for "pushing" evaluation from one or more outputs of a single node. This causes the "pushed" outputs to begin evaluation in visit-order of a breadth-first-search that ends when nodes are reached that either 1. only have outputs connecting to nodes that have already been evaluated or 2. have no outputs at all.

2. Pull evaluation. The graph allows for "pulling" evaluation from one or more inputs of a single node. This causes the "pulled" inputs to perform a depth-first search in order to find all connected nodes that either 1. Have no inputs or 2. have inputs that connect to already visited nodes. Once these "starting" nodes are found, evaluation is "pushed" from each of these nodes in the order in which they were visited.

A graph may be used to represent the body of a function for a higher-level node. The inputs and outputs to the higher-level node can be described by using special "inlet" and "outlet" nodes within the graph itself.

Compilation

One possibly useful approach may be to encapsulate each node within its own crate. This may result in a number of benefits:

• Faster compilation - crates are the compilation unit of rust - by encapsulating each node in its own crate, the amount of work necessary to perform each time a graph is modified should be minimal.
• Nodes as Dynamic Libraries see Nodes as dynamic/shared libraries #3.
• Nodes as Static Libraries.
• Leverage rustc - Access to the same level of optimisation as rust itself. Typechecker can be used for generating useful errors/warnings.
• Leverage cargo - simplifies publishing and searching for nodes, workspaces simplify sharing compilation targets for a single project.

Project

A gantz project represents the context in which the user composes their gantz graph together at runtime. Each project either shares an existing cargo workspace or has their own associated cargo workspace, which stores all locally created node crates.

TODO: Need to decide how to handle the modification of nodes in a shared workspace. It may be better to always require a unique workspace per project and rely on some sort of cache like sccache to speed up initial compilation.

Node Sources

Preexisting nodes may be sourced from a variety of locations, similarly to how cargo may source preexisting crates from a variety of locations.

• Crates from crates.io with some given prefix (e.g. gantz-node-*).
• Crates from some local directory specified via a Path.
• Crates from some git repository specified via a Url.

Node Kinds

A node may take one of the following forms:

• Gantz graph: Inlets describe the inputs, outlets describe the outputs and logic/state are described by the composition of the graph between the inlets and outlets.
• Rust Item: Rust items that can be used as an expression, e.g. function where arguments describe the inputs, the output type fields describes the output. Expressions could be stored as the AST that represents the expression along with the full path (starting from the crate name) to access that expression.
• Rust Expression: It should be possible to create a node via a string that can be fed to a quote! macro that results in generating an expression.

Push and Pull Nodes

In order to know how to evaluate the graph we must know how to generate rust code for execution. To know how to generate code for execution, we must know the exact points from which push or pull evaluation might be triggered so that functions may be generated for those potential code paths.

Usage Overview

Create a Project

The first step a user takes when working with gantz is to create a Project instance. The project requires a unique name and will generate two directories:

1. A directory at the specified Path containing portable project information, local builds and other useful project data. If no Path is specified, we could default to ~/gantz/projects/<project_name>
2. A directory within ~/.gantz/projects/<project_name> storing the shared workspace.

Adding Nodes

Upon creation, the project creates an empty Graph which will be used to compose together nodes.

The user may then add Nodes to the graph. Nodes can either be made up of some subgraph or by some rust expression. In practise, it may be useful to provide interfaces for searching node sources (as mentioned above), writing a small rust expression or creating an empty subgraph node to start working with.

Connecting Nodes

Graph compilation does not begin until nodes are connected together and program composition begins. Compilation occurs by generating a function for each possible path of push and pull evaluation through the graph. The crate (or crates?) containing these generated functions is compiled as a dynamic library. The running instance of gantz then loads this dynamic library ready for runtime execution.

WIP