`pyiron_workflow`-independent dict containing workflow info

[samwaseda]

Note: I will work on this myself. You can just tell me what you think @liamhuber

Currently, we have graph_as_dict, which returns a dict containing all the information about the workflow. However, when we try to make semantikon work with any workflow manager, the most natural thing for me is to ask every workflow manager to provide a dictionary that does not require semantikon to depend on the workflow manager. In addition, that would be consistent with the effort that @jan-janssen is making here.

I was initially thinking we should extend @jan-janssen's universal representation of workflows by adding optional args required by semantikon, and potentially also allowing nodes_dict to contain sub-nodes to represent also the Macro nodes. However, I started realizing that this might not be the right way, because:

• There's no consensus on how macro nodes or metadata should be defined
• (Maybe more importantly) @jan-janssen's universal representation specifies the numerical logic of execution and in this regard it is conceptually complete

So my first suggestion is to separate semantikon-dict from the universal representation dict, in order to represent the workflow including the scientific logic.

Coming to the actual content, semantikon would need:

• IO-values
• Edges
• Unique node labels
• IO labels
• Underlying python function
• (Optionally) type hints

The type hints are optional, because semantikon can parse them, but that could be a bit dangerous because it might fail to assign the types to the IO labels. The reason why we need the original python function is because since this PR the function metadata is attached to f._semantikon_metadata, which I would not ask the workflow manager to provide.

Concerning the Macro nodes, I was testing the following code:

@Workflow.wrap.as_function_node
def add_one(a: int):
    result = a + 1
    return result

@Workflow.wrap.as_function_node
def add_two(b: int):
    result = b + 2
    return result

@Workflow.wrap.as_macro_node
def add_three(macro, c: int):
    macro.one = add_one(a=c)
    macro.two = add_two(b=macro.one)
    result = macro.two
    return result

wf = Workflow("my_wf")
wf.three = add_three(c=1)
wf.four = add_one(a=wf.three)
wf.run()

Then I realized that wf.graph_as_dict does explicitly not provide the connection between wf.three.inputs.c and wf.three.one.inputs.a as well wf.three.two.outputs.result and wf.three.outputs.result. They may not be edges in the sense of workflow edges, but they still somehow have to be included in the dict.

Well, thanks for reading, but actually I'm probably the one who profited from it the most as it helped me figure out what I actually want XD

[samwaseda]

Today, I had a discussion with @jan-janssen while jogging, and he sounded he's actually interested in extending the universal representation, and I thought about it now for quite some time for macro nodes. With this in mind, question for @jan-janssen, but also for @liamhuber: On this page @jan-janssen defined the universal representation of the following code:

def add_x_and_y(x, y):
    z = x + y
    return x, y, z

def add_x_and_y_and_z(x, y, z):
    w = x + y + z
    return w

x, y, z = add_x_and_y(x=1, y=2)
w = add_x_and_y_and_z(x=x, y=y, z=z)

via (I defined a dict instead of individual variables for the reasons that will follow, but the content is the same)

workflow_dict = {
    "nodes_dict": {
        0: add_x_and_y,
        1: add_x_and_y_and_z,
        2: 1,
        3: 2,
    },
    
    "edges_lst": [
        {'target': 0, 'targetHandle': 'x', 'source': 2, 'sourceHandle': None},
        {'target': 0, 'targetHandle': 'y', 'source': 3, 'sourceHandle': None},
        {'target': 1, 'targetHandle': 'x', 'source': 0, 'sourceHandle': 'x'},
        {'target': 1, 'targetHandle': 'y', 'source': 0, 'sourceHandle': 'y'},
        {'target': 1, 'targetHandle': 'z', 'source': 0, 'sourceHandle': 'z'},
    ]
}

Now my suggestion: How about changing it to the following?

nodes_dict = {
    0: add_x_and_y,
    1: add_x_and_y_and_z,
}

edges_lst = [
    {'target': 0, 'targetHandle': 'x', 'source': "input", 'sourceHandle': "X", "value": 1},
    {'target': 0, 'targetHandle': 'y', 'source': "input", 'sourceHandle': "Y", "value": 2},
    {'target': 1, 'targetHandle': 'x', 'source': 0, 'sourceHandle': 'x'},
    {'target': 1, 'targetHandle': 'y', 'source': 0, 'sourceHandle': 'y'},
    {'target': 1, 'targetHandle': 'z', 'source': 0, 'sourceHandle': 'z'},
    {'target': "output", 'targetHandle': "W", 'source': 1, 'sourceHandle': 'w'},
]

The reason is fairly easy: Let's say I have the following macro node:

def all_at_once(X, Y):
    x, y, z = add_x_and_y(x=X, y=Y)
    w = add_x_and_y_and_z(x=x, y=y, z=z)
    W = w
    return W

Then I can define:

macro_dict = {
    "nodes_dict": {
        0: add_x_and_y,
        1: add_x_and_y_and_z,
    },
    "edges_lst": [
        {'target': 0, 'targetHandle': 'x', 'source': "input", 'sourceHandle': "X", "value": 1},
        {'target': 0, 'targetHandle': 'y', 'source': "input", 'sourceHandle': "Y", "value": 2},
        {'target': 1, 'targetHandle': 'x', 'source': 0, 'sourceHandle': 'x'},
        {'target': 1, 'targetHandle': 'y', 'source': 0, 'sourceHandle': 'y'},
        {'target': 1, 'targetHandle': 'z', 'source': 0, 'sourceHandle': 'z'},
        {'target': "output", 'targetHandle': "W", 'source': 1, 'sourceHandle': 'w'},
    ]
}

Now let's say we have the following workflow:

w = all_at_once(X=1, Y=2)
x, y, z = add_x_and_y(x=w, y=3)

Then the new definition would allow us to write something like:

workflow_dict = {
    "nodes_dict": {
        0: macro_dict,
        1: add_x_and_y_and_z,
    },
    "edges_lst": [
        {'target': 0, 'targetHandle': 'X', 'source': "input", 'sourceHandle': None, "value": 1},
        {'target': 0, 'targetHandle': 'Y', 'source': "input", 'sourceHandle': None, "value": 2},
        {'target': 1, 'targetHandle': 'y', 'source': "input", 'sourceHandle': None, "value": 3},
        {'target': 1, 'targetHandle': 'x', 'source': 0, 'sourceHandle': 'W'},
        {'target': "output", 'targetHandle': None, 'source': 1, 'sourceHandle': 'x'},
        {'target': "output", 'targetHandle': None, 'source': 1, 'sourceHandle': 'y'},
        {'target': "output", 'targetHandle': None, 'source': 1, 'sourceHandle': 'z'},
    ]
}

From my point of view, this conceptually incorporates macro nodes. Besides, after the execution value can be optionally set to all the edges as well. What do you think?

[samwaseda]

I forgot to mention, but if we replace the enumeration in the node_dict keys by the node labels, with this representation we can extract all the information that I listed above except for the type hints, which we can probably also optionally return.

[jan-janssen]

I like the idea of an output handle, as previously it was not directly clear which property the user is interested in.

In terms of nesting the macro dictionary inside the node dictionary, I am not so happy as I initially wanted to separate the nodes / python functions from the edges / logic. Now with the nested structure both get merged.

Another suggestion would be:

workflow_dict = {
  "mymacro" : {
    "nodes_dict": {
        0: add_x_and_y,
        1: add_x_and_y_and_z,
    },
    "edges_lst": [
        {'target': 0, 'targetHandle': 'x', 'source': "input", 'sourceHandle': "X", "value": 1},
        {'target': 0, 'targetHandle': 'y', 'source': "input", 'sourceHandle': "Y", "value": 2},
        {'target': 1, 'targetHandle': 'x', 'source': 0, 'sourceHandle': 'x'},
        {'target': 1, 'targetHandle': 'y', 'source': 0, 'sourceHandle': 'y'},
        {'target': 1, 'targetHandle': 'z', 'source': 0, 'sourceHandle': 'z'},
        {'target': "output", 'targetHandle': "W", 'source': 1, 'sourceHandle': 'w'},
    ]
  },
  "main": {
    "nodes_dict": {
        0: "mymacro",
        1: add_x_and_y_and_z,
    },
    "edges_lst": [
        {'target': 0, 'targetHandle': 'X', 'source': "input", 'sourceHandle': None, "value": 1},
        {'target': 0, 'targetHandle': 'Y', 'source': "input", 'sourceHandle': None, "value": 2},
        {'target': 1, 'targetHandle': 'y', 'source': "input", 'sourceHandle': None, "value": 3},
        {'target': 1, 'targetHandle': 'x', 'source': 0, 'sourceHandle': 'W'},
        {'target': "output", 'targetHandle': None, 'source': 1, 'sourceHandle': 'x'},
        {'target': "output", 'targetHandle': None, 'source': 1, 'sourceHandle': 'y'},
        {'target': "output", 'targetHandle': None, 'source': 1, 'sourceHandle': 'z'},
    ]
 },
}

[jan-janssen]

Comment on the use of values in the edges. I feel it is dangerous as the user might not realize that two values are related to each other. Let's say I hard code the lattice constant for aluminium in my workflow. When I use it in multiple structure generators, I might define the same value in multiple edges. For somebody using my workflow it is not clear how these values are related.

[liamhuber]

At a high level I have three pieces:

1. Macro subgraphs should be kept isolated from the main graph. Don't connect them. The macro should behave like any other function, it just so happens that in this case it's a "function" that we can look into and represent as a graph. In that way I like @jan-janssen's last example best, but I still feel like there might be too much connection going on.
2. I agree that attaching values to the edges is dangerous. IMO it falls into the general danger category of mixing up the workflow class (recipe) with the workflow instance. Maybe we can find a way to do it, but probably better to first get a representation we're happy with that only handles the recipe, and then think if it's extensible to handle the data too.
3. I find the whole source/handle business very hard to read. I think we can probably improve readability. I'll try and cook up and example of my own before you guys wake up tomorrow.

[liamhuber]

Given

# Function node
def add_x_and_y(x, y):
    z = x + y
    return x, y, z

# Function Node
def add_x_and_y_and_z(x, y, z):
    w = x + y + z
    return w


# Macro node
def all_at_once(X, Y):
    x, y, z = add_x_and_y(x=X, y=Y)
    w = add_x_and_y_and_z(x=x, y=y, z=z)
    W = w
    return W

# Workflow
all_at_once(1, 2)

I would imagine a graph representation like this:

my_first_node = {
    "function": add_x_and_y,
    "inputs": ("x", "y",),  # These can be automatically scraped
    "outputs": ("x", "y", "z"),  # This can be scraped in pyiron_workflow
    # but not in general -- nevertheless we need some way of identifying outputs,
    # so I guess some meta data like this dictionary is necessary to accompany functions
}

my_second_node = {
    "function": add_x_and_y_and_z,
    "inputs": ("x", "y",), 
    "outputs": ("w",),
}

my_macro = {
    "inputs": ("X", "Y",),
    "outputs": ("W",),
    "nodes": {  # Label to callable
        "xy": my_first_node,
        "xyz": my_second_node,
    },
    "edges": (
        ("inputs.X", "xy.inputs.x"),
        ("inputs.Y", "xy.inputs.y"),
        ("xy.outputs.x", "xyz.inputs.x"),
        ("xy.outputs.y", "xyz.inputs.y"),
        ("xy.outputs.z", "xyz.inputs.z"),
        ("xy.outputs.w", "outputs.W"),
    )
}

workflow = {
    "inputs": ("X", "Y",),
    "outputs": ("W",),
    "nodes": {"m": my_macro},
    "edges": (
        ("inputs.X", "m.inputs.X"),
        ("inputs.Y", "m.inputs.Y"),
        ("m.outputs.W", "outputs.W"),
    )
}

Note that the IO is always explicitly defined, and thus macros are able to indicate how their IO communicates with their children, but there is no direct connection between a macro's siblings and its children. (Maybe this is already the case in @jan-janssen's example! I am just having a bit of trouble grokking it.)

Then notice that here even the functions are required to have an explicty "graphy" representation dictionary. In pyiron_workflow we can always scrape all of this. For general python functions it gets harder when it comes to the output. We have tools in pwf to scrape labels from even the symbols of the code in the return function, but you still can't do it generically because what to do if there's more than one return function? So no matter how we slice it, I think we will need to force generic python callables into some more constrained and explicit form. Here I do it with a dictionary that is based on the returned tuple. One advantage here is that if you imagine I wanted to be working with the tuple (x, y, z) in my graph, the same underlying function could get the following representation to accomplish that:

my_first_node = {
    "function": add_x_and_y,
    "inputs": ("x", "y",), 
    "outputs": ("tup"),  # Now the whole return statement is parsed as a single tuple
    # Of course this flexiblity only goes so far -- we couldn't pull out "(x, y), z" 
    # It's all together or all apart
}

Ok, and finally note that we can imagine extending such a terminology to include annotations:

"inputs": ( ("x", None), ("y", u(int, units="meters"),),

or

"inputs": {
    "x": None,
    "y": u(int, units="meters"),
}

So coming back to a high level after a bit more percolation: I like where you guys are going with this. Excluding the fact that I've dropped values, I think we've got basically all the pieces you mentioned in your first post, @samwaseda. I think my favourite thing so far is my example above + the (label, annotation) dictionary for inputs and outputs.

Until I wrote up this example, I hadn't really thought about it in this generic workflow language context, but now I see we really do need to constrain the bottom-most graph callables -- i.e. the root most functions. We certainly can't graph-ize anything, e.g. functions with multiple return branches with different numbers/natures of items would totally sink this approach, so we'd better constrain them.

[samwaseda]

I feel it is dangerous as the user might not realize that two values are related to each other. Let's say I hard code the lattice constant for aluminium in my workflow. When I use it in multiple structure generators, I might define the same value in multiple edges. For somebody using my workflow it is not clear how these values are related.

The whole idea is to represent a workflow that a user has run or is going to run, that means there is no way whether two values are supposed to be the same or accidentally the same, i.e. we could think about:

output_1 = my_function(structure_1)
output_2 = my_function(structure_2)
output_3 = my_function(structure_2)

Let's assume structure_1 and structure_2 happen to be the same. In this case all three inputs are the same, but while the last two are "supposed" to be the same, the first one "happens" to be the same. So actually it might be more dangerous to group them by their values. Instead, I think the user should make another node to explicitly say that the last two should get the same structure, i.e.:

structure = structure_generator(some_input)
output_1 = my_function(structure_1)
output_2 = my_function(structure)
output_3 = my_function(structure)

and include structure_generator into the workflow.

I can live with @liamhuber's representation (not to mention also with @jan-janssen's), except for the fact that I would like to have a straightforward extension to include values, i.e. instead of:

workflow = {
    "inputs": ("X", "Y",),
    "outputs": ("W",),
    "nodes": {"m": my_macro},
    "edges": (
        ("inputs.X", "m.inputs.X"),
        ("inputs.Y", "m.inputs.Y"),
        ("m.outputs.W", "outputs.W"),
    )
}

I would suggest:

workflow = {
    "inputs": ("X", "Y",),
    "outputs": ("W",),
    "nodes": {"m": my_macro},
    "edges": (
        {"input": "inputs.X", "output": "m.inputs.X"},
        {"input": "inputs.Y", "output": "m.inputs.Y"},
        {"input": "m.outputs.W", "output": "outputs.W"},
    )
}

This being said, it might be slightly more difficult for @jan-janssen to accept this representation, because it would involve more parsing when it's read by the workflow manager. This being said, our suggestions are not so different from each other, so I'm gonna make an example for @liamhuber's one for now and open a PR here.

[samwaseda]

Sorry actually since one dictionary should be returned, the representation should look like this:

result = { 
    "my_first_node": {
        "function": add_x_and_y,
        "inputs": ("x", "y",),  # These can be automatically scraped
        "outputs": ("x", "y", "z"),  # This can be scraped in pyiron_workflow
        # but not in general -- nevertheless we need some way of identifying outputs,
        # so I guess some meta data like this dictionary is necessary to accompany functions
    }   

    "my_second_node": {
        "function": add_x_and_y_and_z,
        "inputs": ("x", "y",), 
        "outputs": ("w",),
    }   

    "my_macro": {
        "inputs": ("X", "Y",),
        "outputs": ("W",),
        "edges": (
            {"input": "inputs.X", "output": "my_first_node.inputs.x"},
            {"input": "inputs.Y", "output": "my_first_node.inputs.y"},
            {"input": "my_first_node.outputs.x", "output": "my_second_node.inputs.x"},
            {"input": "my_first_node.outputs.y", "output": "my_second_node.inputs.y"},
            {"input": "my_first_node.outputs.z", "output": "my_second_node.inputs.z"},
            {"input": "my_second_node.outputs.w", "output": "outputs.W"},
        )   
    }   

    "workflow": {
        "inputs": ("X", "Y",),
        "outputs": ("W",),
        "edges": (
            {"input": "inputs.X", "output": "my_macro.inputs.X"},
            {"input": "inputs.Y", "output": "my_macro.inputs.Y"},
            {"input": "my_macro.outputs.W", "output": "outputs.W"},
        )   
    }   
}

I also removed nodes because they are defined by the keys.

[liamhuber]

About to go to sleep, but quickly: I think you probably still want a nodes field, otherwise it's purely implicit which node belongs to which macro/workflow, accessible only by parsing the edges. Might be fine to not require it to specify the dictionary, but for reading the dictionary I'd still like to see these.

[jan-janssen]

For general python functions it gets harder when it comes to the output. We have tools in pwf to scrape labels from even the symbols of the code in the return function, but you still can't do it generically because what to do if there's more than one return function? So no matter how we slice it, I think we will need to force generic python callables into some more constrained and explicit form.

At least for the general workflow definition, I do not expect anybody to write this graph directly. Rather I expect them to use one of the existing workflow frameworks and export the graph in this format. The advantage is that the current workflow frameworks all have a way how to declare the expected output, even though they might not agree to do it in the same way.

[samwaseda]

About to go to sleep, but quickly: I think you probably still want a nodes field, otherwise it's purely implicit which node belongs to which macro/workflow, accessible only by parsing the edges. Might be fine to not require it to specify the dictionary, but for reading the dictionary I'd still like to see these.

That's true... One possibility is to merge the ideas of you two: I changed @jan-janssen's version with node and variable combined

workflow_dict = { 
  "mymacro" : { 
    "nodes_dict": {
        "xy": add_x_and_y,
        "xyz": add_x_and_y_and_z,
    },  
    "edges_lst": [
        {'target': 'xy.inputs.x', 'source': "input.X", "value": 1}, 
        {'target': 'xy.inputs.y', 'source': "input.Y", "value": 2}, 
        {'target': 'xyz.inputs.x', 'source': 'xy.outputs.x'},
        {'target': 'xyz.inputs.y', 'source': 'xy.outputs.y'},
        {'target': 'xyz.inputs.z', 'source': 'xy.outputs.z'},
        {'target': "output.W", 'source': 'xyz.outputs.w'},
    ]   
  },  
  "main": {
    "nodes_dict": {
        "mymacro": "mymacro",
        "xyz": add_x_and_y_and_z,
    },  
    "edges_lst": [
        {'target': 'mymacro.inputs.X', 'source': "input", "value": 1}, 
        {'target': 'mymacro.inputs.Y', 'source': "input", "value": 2}, 
        {'target': 'xyz.inputs.y', 'source': "input", "value": 3}, 
        {'target': 'xyz.inputs.x', 'source': 'mymacro.outputs.W'},
        {'target': "output", 'source': 'xyz.outputs.x'},
        {'target': "output", 'source': 'xyz.outputs.y'},
        {'target': "output", 'source': 'xyz.outputs.z'},
    ]   
 },
}

And I claim this is closer to what pyiron_workflow does than your representation, because in your representation the combination of my_first_node = {"function": add_x_and_y, ...} and my_macro = {... "nodes": {"xy": my_first_node, ...} ...} implies add_x_and_y is first instantiated (to my_first_node) and assigned to my_macro.xy, which is not the case. In reality, add_x_and_y is directly instantiated to my_macro.xy, which is what the representation of @jan-janssen's corresponds to.

[jan-janssen]

I am not bound to a specific notation and for me the two are equivalent:

{'target': 'xyz', 'targetHandle': 'x', 'source': 'xy', 'sourceHandle': 'x'},

and:

{'target': 'xyz.inputs.x', 'source': 'xy.outputs.x'},

so from that side, I am fine with either solution.

The part where I am still not sure, is where to store the values. I could live with using the "value" field of the edges for optional storage of outputs, but at least for the default input values and the parameters the user is able to change, I would prefer those to be nodes.

[samwaseda]

The part where I am still not sure, is where to store the values. I could live with using the "value" field of the edges for optional storage of outputs, but at least for the default input values and the parameters the user is able to change, I would prefer those to be nodes.

I'm not sure if I'm understanding the problem correctly. I was thinking about writing a function like get_dict_representation with a default argument values=False. If values is True and the workflow has already run, then I would put the values to all the edges, otherwise only the default values would be attached, which would correspond to the case that I presented above. That's not good enough?

[jan-janssen]

Does it make sense to have the outside input and output variables specifically marked. I mean when I search for a workflow which calculates a bulk modulus, I do not really want to parse all the edges to see if the bulk modulus is an output edge.