Callbacks cannot remove and add nodes

[agersch2]

I am trying to show a graph with parent and child nodes (representing clusters) and have a menu or button callback to remove/hide the parent nodes and leave the children (and appropriate edges.)
I've tried various approaches, but essentially, they seem to all return the same error.
The callback(s) return a whole new "elements" to Cytoscape and they inevitably result in a JS error in the browser like this example:

Can not create edge 7cf808c5-c6a2-4937-8ca8-8b43d1bd8893 with nonexistant source nyc

I have put together a small test/demo to show the problem, based directly on some of the examples from the documentation.
As far as I can see from the documentation examples, this should do what I want, but I still get the errors. I don't think I'm missing anything (this is pretty straightforwardly copying and pasting from the examples...) so I suspect there's a but here.

In my debugging efforts, I noticed that if one changes the IDs of the nodes in the callback instead of leaving them the same as the originals, then it seems to work. But that would be somewhat kludgey and cumbersome to do for a graph with lots of actual data.
(Note: I did try looking at the JS in the browser console tool. if I followed it correctly, it looks like it's removing all the nodes and edges, then trying to add back edges before creating the nodes... maybe...)

Thanks in advance for your help!
-Al

My Example CODE:

---

import dash
import dash_cytoscape as cyto
import dash_html_components as html
from dash.dependencies import Input, Output, State

app = dash.Dash(name)
app.config.suppress_callback_exceptions = True

app.layout = html.Div([
cyto.Cytoscape(
id='cytoscape-compound',
layout={'name': 'preset'},
style={'width': '100%', 'height': '450px'},
stylesheet=[
{
'selector': 'node',
'style': {'content': 'data(label)'}
},
{
'selector': '.countries',
'style': {'width': 5, 'visible': 'false' }
},
{
'selector': '.cities',
'style': {'line-style': 'dashed'}
}
],
elements=[
# Parent Nodes
{
'data': {'id': 'us', 'label': 'United States'}
},
{
'data': {'id': 'can', 'label': 'Canada'}
},
# Children Nodes
{
'data': {'id': 'nyc', 'label': 'New York', 'parent': 'us'},
'position': {'x': 100, 'y': 100}
},
{
'data': {'id': 'sf', 'label': 'San Francisco', 'parent': 'us'},
'position': {'x': 100, 'y': 200}
},
{
'data': {'id': 'mtl', 'label': 'Montreal', 'parent': 'can'},
'position': {'x': 400, 'y': 100}
},
# Edges
{
'data': {'source': 'can', 'target': 'us'},
'classes': 'countries'
},
{
'data': {'source': 'nyc', 'target': 'sf'},
'classes': 'cities'
},
{
'data': {'source': 'sf', 'target': 'mtl'},
'classes': 'cities'
}
]
),
html.Div([
html.Button('Add', id='btn-add-node-example', n_clicks_timestamp=0),
html.Button('Remove', id='btn-remove-node-example', n_clicks_timestamp=0)
])
])

@app.callback(Output('cytoscape-compound', 'elements'),
[Input('btn-add-node-example', 'n_clicks_timestamp'),
Input('btn-remove-node-example', 'n_clicks_timestamp')],
[State('cytoscape-compound', 'elements')])
def update_elements(btn_add, btn_remove, elements):
if int(btn_add) > int(btn_remove):
elements = [
# Children Nodes
{
'data': {'id': 'nyc', 'label': 'New York', 'parent': 'us'},
'position': {'x': 100, 'y': 100}
},
{
'data': {'id': 'sf', 'label': 'San Francisco', 'parent': 'us'},
'position': {'x': 100, 'y': 200}
},
{
'data': {'id': 'mtl', 'label': 'Montreal', 'parent': 'can'},
'position': {'x': 400, 'y': 100}
},
# Parent Nodes
{
'data': {'id': 'us', 'label': 'United States'}
},
{
'data': {'id': 'can', 'label': 'Canada'}
},

        # Edges
        {
            'data': {'source': 'can', 'target': 'us'},
            'classes': 'countries'
        },
        {
            'data': {'source': 'nyc', 'target': 'sf'},
            'classes': 'cities'
        },
        {
            'data': {'source': 'sf', 'target': 'mtl'},
            'classes': 'cities'
        }
    ]
    return elements
elif int(btn_remove) > int(btn_add):
    elements = [
        {
            'data': {'id': 'nyc', 'label': 'New York'},
            'position': {'x': 100, 'y': 100}
        },
        {
            'data': {'id': 'sf', 'label': 'San Francisco'},
            'position': {'x': 100, 'y': 200}
        },
        {
            'data': {'id': 'mtl', 'label': 'Montreal'},
            'position': {'x': 400, 'y': 100}
        },
        # Edges
        {
            'data': {'source': 'nyc', 'target': 'sf'},
            'classes': 'cities'
        },
        {
            'data': {'source': 'sf', 'target': 'mtl'},
            'classes': 'cities'
        }
    ]
    return elements
return elements

if name == 'main':
app.run_server(debug=True)

[agersch2]

UPDATE:

Actually, it's not just parent and child nodes...

When I tried copying the add/remove example code (exactly, no changes) and running it locally, I get (many) similar errors - when clicking on the add (back after removing) button!!!
https://dash.plotly.com/cytoscape/callbacks
And I now note that it doesn't even work on the demo/example page either!!!

import dash
import dash_cytoscape as cyto
import dash_html_components as html
import dash_core_components as dcc
from pprint import pprint
from dash.dependencies import Input, Output, State

app = dash.Dash(name)

nodes = [
{
'data': {'id': short, 'label': label},
'position': {'x': 20lat, 'y': -20long}
}
for short, label, long, lat in (
('la', 'Los Angeles', 34.03, -118.25),
('nyc', 'New York', 40.71, -74),
('to', 'Toronto', 43.65, -79.38),
('mtl', 'Montreal', 45.50, -73.57),
('van', 'Vancouver', 49.28, -123.12),
('chi', 'Chicago', 41.88, -87.63),
('bos', 'Boston', 42.36, -71.06),
('hou', 'Houston', 29.76, -95.37)
)
]

edges = [
{'data': {'source': source, 'target': target}}
for source, target in (
('van', 'la'),
('la', 'chi'),
('hou', 'chi'),
('to', 'mtl'),
('mtl', 'bos'),
('nyc', 'bos'),
('to', 'hou'),
('to', 'nyc'),
('la', 'nyc'),
('nyc', 'bos')
)
]

default_stylesheet = [
{
'selector': 'node',
'style': {
'background-color': '#BFD7B5',
'label': 'data(label)'
}
},
{
'selector': 'edge',
'style': {
'line-color': '#A3C4BC'
}
}
]

app.layout = html.Div([
html.Div([
html.Button('Add Node', id='btn-add-node', n_clicks_timestamp=0),
html.Button('Remove Node', id='btn-remove-node', n_clicks_timestamp=0)
]),

cyto.Cytoscape(
    id='cytoscape-elements-callbacks',
    layout={'name': 'circle'},
    stylesheet=default_stylesheet,
    style={'width': '100%', 'height': '450px'},
    elements=edges+nodes
)

])

@app.callback(Output('cytoscape-elements-callbacks', 'elements'),
[Input('btn-add-node', 'n_clicks_timestamp'),
Input('btn-remove-node', 'n_clicks_timestamp')],
[State('cytoscape-elements-callbacks', 'elements')])
def update_elements(btn_add, btn_remove, elements):
# If the add button was clicked most recently
if int(btn_add) > int(btn_remove):
next_node_idx = len(elements) - len(edges)

    # As long as we have not reached the max number of nodes, we add them
    # to the cytoscape elements
    if next_node_idx < len(nodes):
        return edges + nodes[:next_node_idx+1]

# If the remove button was clicked most recently
elif int(btn_remove) > int(btn_add):
    if len(elements) > len(edges):
        return elements[:-1]

# Neither have been clicked yet (or fallback condition)
return elements

if name == 'main':
app.run_server(debug=True)

[agersch2]

Also, please see here:
https://github.com/plotly/dash-docs/blob/b2a8ab2e3b35483ac625a0df0214e21a41dffe38/tutorial/examples/cytoscape/elements_callbacks.py

[riverchen99]

Running into this issue as well. Trying to do a similar thing -- removing the "parent" property from the data dictionary while keeping same IDs throws that error for me.

This section of the underlying Cytoscape.js API states that the parent/child relationship is immutable.

Not sure if possible, but it would be great if Dash Cytoscape could expose a way to access the underlying Cytoscape.js API to allow users to use methods like eles.move().

[riverchen99]

@agersch2 I found a workaround to this issue.

When modifying the parent/child relationship of a node, or the source/target of an edge (with id), I append a unique UUID string to the ID field of each element. This forces a rerender which breaks the animation feature, but works to display the intended output.

I keep the UUID of the previous set of elements if the callback only changes the class of the element without touching the data dictionary.

[agersch2]

@riverchen99 Thanks for the commiseration and helpful suggestion.
I had already done a similar - very kludgey - workaround. But I'd prefer a real solution to the issue to a workaround.
Also, as mentioned above, it's not only parent-child node modifications that don't work. Removing and adding nodes exactly per the documentation example doesn't either! I think that requires a real fix.

[xhluca]

@agersch2 @riverchen99 This definitely seems like an issue in v0.2.0, since the add/remove node example worked correctly with older versions of dash-cytoscape (v0.1.1 and v0.0.4). I'll spend some time investigating this issue and hopefully find a fix for v0.2.1.

[xhluca]

@agersch2 I'm having copy pasting your code because of indentation. Could you share a gist or use markdown formatting? You can do something like this:

```python
paste your code here
```

Thank you.

[xhluca]

It seems like it was introduced in: #92

@mj3cheun Do you have an idea whether it's an intentional behavior of cyResponsiveGraph? I'm re-reading the cyResponsive.js code and I can't immediately see how it could affect the behavior of adding/removing sourceless or targetless edges. Would you have an idea why this would be happening?

I've added some steps for you to verify if you have the bandwidth to take a look at this.

---

Run this gist with dash-cytoscape before and after the PR (#92). To do so:

1. git clone https://github.com/plotly/dash-cytoscape
2. cd dash-cytoscape
3. save the gist as app.py
4. checkout to the working/non-working commits (see below)
5. python app.py

Working commit: git checkout 4511b8915ad6673213ff4b161bdc1e3a3bc9b5a2
Non-working commit: git checkout 4cba5bcc8aa8110b5365c0995e177b318e09c18d

[agersch2]

import dash
import dash_cytoscape as cyto
import dash_html_components as html
import dash_core_components as dcc
from dash.dependencies import Input, Output, State

app = dash.Dash(__name__)
app.config.suppress_callback_exceptions = True

app.layout = html.Div([
    html.Div([
        html.Button('Add', id='btn-add-node-example', n_clicks_timestamp=0),
        html.Button('Remove', id='btn-remove-node-example', n_clicks_timestamp=0)
    ]),
    cyto.Cytoscape(
        id='cytoscape-compound',
        layout={'name': 'preset'},
        style={'width': '100%', 'height': '450px'},
        stylesheet=[
            {
                'selector': 'node',
                'style': {'content': 'data(label)'}
            },
            {
                'selector': '.countries',
                'style': {'width': 5, 'visible': 'false' }
            },
            {
                'selector': '.cities',
                'style': {'line-style': 'dashed'}
            }
        ],
        autoRefreshLayout=True,
        elements=[
            # Parent Nodes
            {
                'data': {'id': 'us', 'label': 'United States'}
            },
            {
                'data': {'id': 'can', 'label': 'Canada'}
            },
            # Children Nodes
            {
                'data': {'id': 'nyc', 'label': 'New York', 'parent': 'us'},
                'position': {'x': 100, 'y': 100}
            },
            {
                'data': {'id': 'sf', 'label': 'San Francisco', 'parent': 'us'},
                'position': {'x': 100, 'y': 200}
            },
            {
                'data': {'id': 'mtl', 'label': 'Montreal', 'parent': 'can'},
                'position': {'x': 400, 'y': 100}
            },
            # Edges
            {
                'data': {'source': 'can', 'target': 'us'},
                'classes': 'countries'
            },
            {
                'data': {'source': 'nyc', 'target': 'sf'},
                'classes': 'cities'
            },
            {
                'data': {'source': 'sf', 'target': 'mtl'},
                'classes': 'cities'
            }
        ]
    )
    ])

@app.callback(Output('cytoscape-compound', 'elements'),
              [Input('btn-add-node-example', 'n_clicks_timestamp'),
               Input('btn-remove-node-example', 'n_clicks_timestamp')],
              [State('cytoscape-compound', 'elements')])
def update_elements(btn_add, btn_remove, elements):
    if int(btn_add) > int(btn_remove):
        elements = [
            # Children Nodes
            {
                'data': {'id': 'nyc', 'label': 'New York', 'parent': 'us'},
                'position': {'x': 100, 'y': 100}
            },
            {
                'data': {'id': 'sf', 'label': 'San Francisco', 'parent': 'us'},
                'position': {'x': 100, 'y': 200}
            },
            {
                'data': {'id': 'mtl', 'label': 'Montreal', 'parent': 'can'},
                'position': {'x': 400, 'y': 100}
            },
            # Parent Nodes
            {
                'data': {'id': 'us', 'label': 'United States'}
            },
            {
                'data': {'id': 'can', 'label': 'Canada'}
            },

            # Edges
            {
                'data': {'source': 'can', 'target': 'us'},
                'classes': 'countries'
            },
            {
                'data': {'source': 'nyc', 'target': 'sf'},
                'classes': 'cities'
            },
            {
                'data': {'source': 'sf', 'target': 'mtl'},
                'classes': 'cities'
            }
        ]
        return elements
    elif int(btn_remove) > int(btn_add):
        elements = [
            {
                'data': {'id': 'nyc', 'label': 'New York', 'isOrphan': True },
                'position': {'x': 100, 'y': 100}
            },
            {
                'data': {'id': 'sf', 'label': 'San Francisco', 'isOrphan': True},
                'position': {'x': 100, 'y': 200}
            },
            {
                'data': {'id': 'mtl', 'label': 'Montreal', 'isOrphan': True},
                'position': {'x': 400, 'y': 100}
            },
            # Edges
            {
                'data': {'source': 'nyc', 'target': 'sf', 'isOrphan': True},
                'classes': 'cities'
            },
            {
                'data': {'source': 'sf', 'target': 'mtl', 'isOrphan': True},
                'classes': 'cities'
            }
        ]
        return elements
    return elements

if __name__ == '__main__':
    app.run_server(debug=True)

[agersch2]

OK; how's that code look now?

[xhluca]

Thank you for formatting the code! I tried running it on my side with both version 0.2.0 and v0.1.1. It seems indeed that v0.2.0 breaks with this example, and it is only partially working with v0.1.1:

My hypothesis is that the problem happens when we try to use compound nodes (which was pointed out earlier by @riverchen99), since by removing the children-parent dependencies and changing significantly the edges connecting each nodes, I was able to get the following result:

The code is here (it works with both v0.1.1 and v0.2.0):

import dash
import dash_cytoscape as cyto
import dash_html_components as html
import dash_core_components as dcc
from dash.dependencies import Input, Output, State

app = dash.Dash(__name__)


default_elements = [
    # Children Nodes
    {
        'data': {'id': 'nyc', 'label': 'New York'},
        'position': {'x': 100, 'y': 100}
    },
    {
        'data': {'id': 'sf', 'label': 'San Francisco'},
        'position': {'x': 100, 'y': 200}
    },
    {
        'data': {'id': 'mtl', 'label': 'Montreal'},
        'position': {'x': 400, 'y': 100}
    },
    # Parent Nodes
    {
        'data': {'id': 'us', 'label': 'United States'},
        'position': {'x': 200, 'y': 200}
    },
    {
        'data': {'id': 'can', 'label': 'Canada'},
        'position': {'x': 200, 'y': 300}
    },

    # Edges
    {
        'data': {'source': 'can', 'target': 'us'},
        'classes': 'countries'
    },
    {
        'data': {'source': 'nyc', 'target': 'sf'},
        'classes': 'cities'
    },
    {
        'data': {'source': 'sf', 'target': 'mtl'},
        'classes': 'cities'
    }
]


app.layout = html.Div([
    html.Div([
        html.Button('Add', id='btn-add-node-example', n_clicks_timestamp=0),
        html.Button('Remove', id='btn-remove-node-example', n_clicks_timestamp=0)
    ]),
    cyto.Cytoscape(
        id='cytoscape-compound',
        layout={'name': 'preset'},
        style={'width': '100%', 'height': '450px'},
        stylesheet=[
            {
                'selector': 'node',
                'style': {'content': 'data(label)'}
            },
            {
                'selector': '.countries',
                'style': {'width': 5, 'visible': 'false' }
            },
            {
                'selector': '.cities',
                'style': {'line-style': 'dashed'}
            }
        ],
        elements=default_elements
    )
    ])

@app.callback(Output('cytoscape-compound', 'elements'),
              [Input('btn-add-node-example', 'n_clicks_timestamp'),
               Input('btn-remove-node-example', 'n_clicks_timestamp')],
              [State('cytoscape-compound', 'elements')])
def update_elements(btn_add, btn_remove, elements):
    if int(btn_add) > int(btn_remove):
        return default_elements
    elif int(btn_remove) > int(btn_add):
        elements = [
            {
                'data': {'id': 'nyc', 'label': 'New York' },
                'position': {'x': 100, 'y': 100}
            },
            {
                'data': {'id': 'sf', 'label': 'San Francisco'},
                'position': {'x': 100, 'y': 200}
            },
            {
                'data': {'id': 'mtl', 'label': 'Montreal'},
                'position': {'x': 400, 'y': 100}
            },
            {
                'data': {'id': 'us', 'label': 'United States'},
                'position': {'x': 200, 'y': 200}
            },
            # Edges
            {
                'data': {'source': 'nyc', 'target': 'mtl'},
                'classes': 'cities'
            },
            {
                'data': {'source': 'sf', 'target': 'us'}
            }
        ]
        return elements
    return elements

if __name__ == '__main__':
    app.run_server(debug=True)