[Feature Requests] UI component for adding panes to compare plots in an env

[TheShadow29]

Suppose I already have two different plots, say of Metric1 and Metric2. Now I want to plot both of these in the same graph. To my knowledge, one would have to do this in the python file itself. However, it would be nice to have a quick comparison between the two graphs without having to change the python code

A simple add_graph like option in visdom gui should suffice and taking values from the two different graphs.

[rajatvd]

I've written a package which somewhat accomplishes this, but it relies on having a metrics.json file from which to plot graphs. It also integrates with the sacred package. I'm currently working on trying to make it a bit more independent from this.

If this doesn't fit your needs exactly, you could try using the properties pane and write up a callback to do what you require. Have a look at the demo_properties.py example if this isn't clear.

Another approach is to have the two plots in different envs with the same window title, and using the compare envs feature to plot both the metrics on the same graph.

[TheShadow29]

Seems like properites_callback is worth digging into. The compare env wouldn't be good as it would require one to have the same variable name for all the metrics. Your package also looks good, I will definitely refer to it. Thank you. However, I still believe this is something that should be supported natively in visdom.

[JackUrb]

@TheShadow29 - I could see something like this being useful, and would love to have it as a feature, however I don't necessarily have the bandwidth to implement it right now, as it is somewhat involved. If you're interested in helping out, I have a plan in mind for how it'd be implemented. Also open to other ideas on implementation though.

1. There needs to be some kind of UI component that invokes adding windows from the frontend. This should likely be hidden behind a dialog to prevent from clogging the UI
2. Events to create elements should be logged to the server, so that they'll persist on refresh
3. A new event needs to be added to the server to create a ComparePane.
4. A frontend ComparePane needs to be created, which needs access to all of the trace names of all of the other plots. That list will likely need to be maintained by main.js so it's not duplicated if you have multiple compare panes. From this pane you need to be able to select traces to compare.
5. Trace comparisons send an event back to the server, which push the updates back to the frontend. Reiterating that the state for one of these ComparePanes should be maintained on the server.
6. ComparePanes on the frontend that have metrics selected will need to pull the requested traces from different sets of plots and create a layout with those. This pane also needs to subscribe to update events on any of the plots that it is comparing.

[TheShadow29]

@JackUrb Thank you for the detailed roadmap. Definitely seems worth implementing. I can give it a try however, I too have a bit of limited bandwidth. I will definitely try but no promises.

A couple of clarifications:

1. Add Chart option in the UI should be good enough. It should have the option to select which variable names to plot with checkboxes.
2. I am not sure what ComparePane is. Is it the name of the new pane to be created?
3. Did you mean main.json instead of main.js?
4. I am having a bit of trouble understanding the terms events, trace, layout. I am guessing (please correct me if I am wrong) events: is the Callback events namely close, keypress and propertyupdate, and trace is a dictionary containing the data points, and has graph properties, finally layoutis the title of the graph, and axes names. I am not sure where can I get these values from. layout.json is in $HOME/.visdom/view/layout.json. But where do I get the traces from.

Thanks

[JackUrb]

1. An add chart option would be sufficient for now, I was just thinking ahead to when others might want additional frontend widgets like stickies or whatever really. Perhaps you can use glyphicon glyphicon-stats and give it the hover text "Add compare pane" to avoid cluttering the nav bar. Making it so that you only select trace names to plot on creation is a fair choice and reduces some complexity down the line.
2. ComparePane would be what you'll end up creating. It's essentially a PlotPane that handles formatting its own content and data given an array of traces it should be comparing.
3. main.js is the file that handles distributing the contents of the json file to the windows. It'll be important to have a way to associate traces and windows, all of which eventually pass through the processPane function: https://github.com/facebookresearch/visdom/blob/2ef12d43536360405c5a065e9f3eed282b8a76bf/js/main.js#L193
4. You're correct on events and the content of trace and layout, those pieces of data are maintained in the panes object in the main.js file, and the contents are passed along to be rendered in PANES here in the render function. The contents of the pane object are all passed along via the {...pane}. This would mean that upon creating a compare pane you'd have to fill the pane object in panes with the required content so that it can be rendered properly here.