[JEP 0028] Clearly define the smallest unit that is a "Jupyter Server"

[Zsailer]

From JEP 0028 (Jupyter Server):

Separate jupyter_server repository

The server-notebook split is an opportunity to clearly define Jupyter's core services. The old notebook server was comprised of many services, and it could be extended by separate "server extensions". However, it isn't clear what should be a core service versus a server extension. Some extensions were "grand-fathered" in as core services to make the notebook application more full-featured (e.g. nbconvert, terminals, contents, ...). Now that we are separating the server from the classic notebook, we need to reevaluate what services are core to the Jupyter Server.

Jupyter server aims to be a core building block for other applications (like nteract, lab, dashboards, standalone widgets, etc.). To achieve this, we need to define the simplest "unit" that defines a Jupyter Server: kernels, kernelspec, and sessions. Other services become extensions to the core server (using the load_jupyter_server_extension mechanism defined below).

Opening this issue for further discussion.

[Zsailer]

Here is previous conversation around this section from PR #28

@rgbkrk: Core services should stick around, otherwise I think we're going to end up further in this diaspora of endpoints with no concrete aim of what makes up the core of jupyter.

@Zsailer: My original motivation for exploring the idea of merging services and extensions came from projects like kernel_gateway and enterprise_gateway. These projects only provide kernels as a service and drop the other services. My thought was that we could make "which services are provided" a configurable option.

By doing this though, I recognize that I'm approaching a point where the jupyter server is just a configurable tornado server. Perhaps this is the wrong move.

Maybe we need to define what the core jupyter server is--and maybe that's what we have already (minus the notebook). Or maybe "kernels as a service" is the smallest "jupyter unit", and other services should be extensions?

@rgbkrk: I'd think kernels and kernelspecs at the very least, as well as sessions.

I understand not having contents be part of this core service, though maybe that makes sessions non-functional(?).

@kevin-bates: @Zsailer - thank you for writing this proposal and starting this great discussion!

@rgbkrk - I totally agree with the notion that kernel management (which needs to include kernelspecs or providers, etc. as you noted) are the root of what "a jupyter server" provides. It's not clear to me what sessions actually provide from a server perspective since clients of the gateways don't need them to spawn kernels.

Although I'm biased, I believe the ability to optionally (and easily) route the kernel management to another node is crucial for multi-tenant capabilities and achieves a natural separation of user data from computation. I can't think of good reasons to necessarily separate out other pieces but that could be due to my lack of experience outside of the gateways. I'm not saying that jupyter_server should solely consist of kernel management - not at all. Just that the "server" should be able to optionally remote certain services. One service that makes sense for this is kernel management (and by association kernelspec management).

@lresende fyi

[vidartf]

In my view there are two different (but related) discussions here:

1. Should jupyter_server remove some services from its default set? These services would then have to be enabled explicitly by inheriting apps (or user config).
2. Should jupyter_server's default set of services be overridable? I.e. can inherting apps replace the default set with a smaller, or alternative set of services.

I think the line between these two has been a bit blurry in this discussion.

In my view:

• Pt 1. is a longer discussion that are fundamental to what constitutes a server app.
• Pt. 2 is a potentially shorter discussion, and enables apps with specific needs to reuse jupyter code without vendoring the current code.

Is this issue about pt. 1, pt. 2, or both?

[Zsailer]

Thanks @vidartf. These are some great discussion points.

How would you answer the following question?

"How should other applications interface with Jupyter Server?" (see issue jupyter-server/jupyter_server#29)

My answer is that the ServerApp shouldn't be "inherited" by other apps. Rather, it should only be extended using server extensions using a load_jupyter_server_extension function or the ExtensionApp (that's what I did with Voila in voila-dashboards/voila#270 ). The ServerApp is the core "atom"/fundamental unit that makes up a Jupyter-based application. In this view, everything else is an extension.

I don't think we should allow such extensions to "turn off" server handlers. Removing server handlers this way would lead to confusion around the server. If an extension needs to remove some service/handler, that indicates to me that it shouldn't be a core, promised service—we should make it an extension instead.

I'm worried pt. 2 is a temporary patch that will introduce backwards incompatibility (or buggy future shims) later.

[kevin-bates]

I think we may want to distinguish between services and their handlers in this discussion. I view the handlers as the entities that expose a service to users (aka the "promise"). For example, w/o the content handlers the content service is not available to external applications. However, the content service is still useful from within the server for use from other services. This may have been what @rgbkrk was driving at regarding his "Core services should stick around..." comment. As such, I think we need to allow an ExtensionApp to disable handlers.

If instead we wanted to have a "service composition" capability, I still think there needs to be a way to disable the handlers - unless we go down the road of a full blown micro services approach where access to the service is only available via handlers (which I don't think we want or need).

Perhaps it might be helpful to first decide what "services" should be included the in server, then ask about their handlers (although I believe one could argue that all handlers should be able to be disabled). Looking at the services sub-package, entries that seem questionable are nbconvert (both handlers and code) and contents (only handlers per my example above). Seems like nbconvert is a good example of moving it to an extension (but I might be blowing smoke just as well). api, config and sessions seems like they'd have application even in base server scenarios. Can't really tell the usefulness of security but it looks to be just exposing a report, so could also have merit in the base scenario. (Btw, it seems like ExtensionApps and server extensions should be able to 'add' their api such that its exposed by the api handler.)

[vidartf]

My answer is that the ServerApp shouldn't be "inherited" by other apps.

I agree that people shouldn't need to do that in order to get the capabilities they need. However, if that is the only way to achieve what they want, they will. Pt. 2 is mostly meant as an escape hatch for those cases where you can't do what you want from an extension/ExtensionApp.

I don't think we should allow such extensions to "turn off" server handlers. Removing server handlers this way would lead to confusion around the server. If an extension needs to remove some service/handler, that indicates to me that it shouldn't be a core, promised service—we should make it an extension instead.

(my emphasis added)

If that is the selection criteria, then it will likely reduce down to only kernel/kernelspec and probably session handlers.

We could possibly also solve this by layering: ServerApp having no/minimum handlers, and a JupyterServerApp (or some other creative name pair) having the current full set of handlers.

[Zsailer]

@vidartf and @kevin-bates Great stuff here! Thank you both for doing the deep dive and opening up this conversation.

Let me think about this over the weekend and I'll post here again with some follow up thoughts.

[vidartf]

nudge ;)

[bollwyvl]

I think the smallest unit depends on who you are. we have a few traitornado products that are logically decomposable into smaller pieces. - notebook: conf + extensions + contents + convert + specs + kernelspecs + sessions + kernels - nbviewer: conf + contents + convert + cache - jupyterhub: conf + auth + env + services - lab: notebook + js build Some stuff we _still_ don't have natively, but basically all of those could benefit from: - search - graph store - localization (vs checked in translations) - version control (or at least greater awareness) - live API documentation - general documentation serving Getting to reuse more pieces across all of those would be really powerful: - nbviewer + auth: often requested for a long time - hub + js build: would ease some challenges we're seeing in serving more tailored apps - notebook + envs: spawn@home! On the traits/tornado aspect: we have to do a lot of stuff ourselves, and our stuff doesn't work like other peoples' stuff. People can't embed our stuff. We can't easily embed other people's stuff (though async is helping). The ASGI spec... https://asgi.readthedocs.io ... came out of Django, but has seen good adoption across a broader swathe of projects. I'd love to do/see a breakdown of our products into this space, but would be more (verifiably) composable. To achieve that, things like openapi, more json schema, type checking, and even graphql, would help a lot. We tend not to (need to) worry/measure _too_ much performance, but probably should do more, at this sub-unit level, as it leaves other people to do it when they integrate (or choose not to) our stuff. Http/2 isn't great for the standalone case, because https, but would be great in basically every other case... And we might be in the position to encourage people to learn about https/certs, and even 2fa. But I don't see some of this materializing in tornado. So at the end of the day, smaller, more tested/quantified, more spec'ed, more reusable units might be the answer, after you ask the second question of " what do you want to do?" Some things I've been researching: - ASGI, schema, openapi: - https://github.com/tiangolo/fastapi GraphQL: - https://github.com/tartiflette/tartiflette - https://github.com/mirumee/ariadne - http2: - https://gitlab.com/pgjones/hypercorn - perf/testing - https://github.com/airspeed-velocity/asv - https://github.com/locustio/locust - https://github.com/HypothesisWorks/hypothesis

[Zsailer]

we have a few traitornado products that are logically decomposable into
smaller pieces.

@bollwyvl these examples make a good point (that @kevin-bates and @vidartf were making too, I was just being stubborn 😆). It doesn't matter what we choose as the "smallest unit", someone will compose their own server with whatever handlers they want. We're doing it ourselves!

Thanks, @kevin-bates for distinguishing handlers + services. You're right, blending them was causing an issue on my thinking. I think adding a default_handlers attribute to jupyter_server that ServerApp subclasses can override is a good idea. I think we can make this change now, without having to answer the bigger question of "what belongs as a core service?".

So at the end of the day, smaller, more tested/quantified, more spec'ed, more reusable units might be the answer, after you ask the second question of " what do you want to do?"

Thanks, @bollwyvl. I absolutely love all these ideas. We should chat over video to discuss further. Do you have time in the coming days to chat?

This sounds like a complete rewrite of the jupyter_server codebase (though, I admit I haven't done enough research on these projects to be sure). I'm not opposed to that (actually I secretly would love this), but maybe that's something we do in parallel? Looking at tools like ASGI, schemas, and openapi, I bet we could quickly rebuild the jupyter_server from scratch and gain a lot of these things you mentioned. The separation of server and notebook allows us to develop this type of "new implementation of the server" and swap it out later without interruption for users (?), especially is this proves to be fruitful. I'd love to work on this with you (and anyone else).

[Zsailer]

Closing due to inactivity. A lot of these ideas were picked up by jupyverse.