Where to next with dashboards, and dashboard server in particular?

[parente]

Members of the Jupyter community need to know where dashboard capability in Jupyter is headed so that they can plan appropriately. Can we put our heads together and make a clear statement about what's next, overall and at the project level?

I'll try to kick off the discussion by documenting where things stand across the projects. It might take me a few editing sessions to finish. I'll cc folks when the text here is substantially complete.

State of affairs

• The dashboards layout extension lets users drag/drop Jupyter notebook output cells into layouts. That is its entire scope, and so the code is relatively stable. The doc for the project outlines the layout metadata format stored in notebook documents which should allow for dashboard rendering in other tools.
• The dashboards server project lets users deploy notebook-defined layouts as standalone web apps. Supporting this capability for notebooks with arbitrary frontend and backend dependencies is a really hard problem with many unstable dependencies.
• The dashboards bundler notebook extension lets users one-click notebook documents and some frontend web assets from a notebook server to a dashboard server. It will soon depend on the bundler API in notebook 5.0. It is otherwise pretty simple and stable.
• JupyterLab has started collecting use cases for dashboarding support in the new UX: Dashboarding jupyterlab/jupyterlab#1640.
• The limited cycles @parente gets to contribute to Jupyter are more focused on services at the moment (e.g., docker-stacks, nbviewer, kernel gateway).

Challenges Regarding the Dashboard Server

• Notebooks let people write both front and backend code that can do pretty much anything in the browser and with the kernel. Providing a one-click-to-deploy experience for any notebook with any dependencies is untenable from a purely open source standpoint. The README tries to call this out in terms of what libraries are known to work, but admittedly does a mediocre job and I haven't gotten back to making it better in Improve project scope defined in README #302.
• If the scope of supported libraries and languages is meant to be limited out of the box, then some mechanism has to be provided to let others extend the capabilities. Inventing yet another Jupyter extension system is a major undertaking, and seems redundant in light of the work being done on Lab and how it may wind up supporting dashboards in the future. Documenting the manual process like in jupyter-matplotlib 404 #301 or accepting pull requests like nteract does to support new renders is probably best for now, but sets a high technical bar.
• The dashboard server uses JupyterLab components for rendering notebook outputs. JupyterLab is still rapidly evolving toward a 1.0. Staying abreast of the API changes in the components requires continual effort.
• The current implementation has to support the deployment of notebooks that were created in the classic notebook UI while using Lab components under the covers. Trying to get the behavior of libraries that do not officially support Lab yet to work properly and look-and-feel like they do in the original classic notebook requires continual effort across versions.

Still more soon, but adding a few cc's ...

/cc @blink1073 @jasongrout @willingc @fperez

[rserbitar]

@parente, thanks for this effort.

I just want to chime in and say that jupyter_dashboards_server along with the layout extension and bundler, provides exactly what is needed. The ability to deploy interactive dashboards with a click of a button to people who dont have jupyter installed locally and do not want to see the code, in a way that looks exactly like the notebook (with the additional extra of getting the ability to do some layouting).

What we basically need is to show others the content of a notebook (including interactivity) without the code. dashboard_servers does this perfectly, and I am concerned to see it get so little attention in the last 8 month (especially after promoting it in the IBM blog first).

[cbcunc]

Good discussion at SciPy about this. More of the Jupyter team starting to understand that it's not just about dashboards for interactive exploration, but also scientists deploying dashboard apps.

[jasongrout]

Yes, one-click deploying of dashboard apps has been in discussion for a long time.

[tylere]

@cbcunc would you be able to provide a summary of the SciPy discussions on deployable dashboards?