[v3-future] Enhance and generalize secret/key management?

[krassowski]

• The API keys set on the UI need to be set again in the kernel for %ai magics to work
• There is a larger interest in having a general Secrets manager, mostly driven by the move to remote APIs for LLM interactions (beyond usage of jupyter-ai alone)

As in my earlier comment:

More generally JupyterLab, could benefit from having a first-class citizen Credentials Store. The Eugo team (@sanjay1890 and @BwL1289) recently brought up to my attention that Google Colab has a Secrets tab in the sidebar:

and that there is a (non-maintained) https://github.com/frankzickert/jupyterlab-credentialstore extension for JupyterLab implementing a similar approach.

We could pass the secrets via environment variables (we already pass JPY_SESSION_NAME down to the kernel to let user now the notebook name).

It feels like moving some of that logic out of jupyter-ai would be a good investment because we would have only one place where we need to validate the logic for security (rather than multiple implementations). The benefits of reuse would seem higher than in the current effort to reuse the chat component.

Any thoughts?

Originally posted by @krassowski in #1103

While I would stop short of proposing to make a new repo like jupyter-sercets, I wonder if shifting the code within the monorepo to a new package to create a "Key Manager" and making it a server extension with a way to pass keys as environment variables to kernels (and possible a UI for adding more secrets) could be in scope of jupyter-ai?

[dlqqq]

@krassowski I agree strongly that there is a need for a general secrets manager in Jupyter. This seems like a good subject to raise to the SSC and the Security Council. I'll add a note to bring this up in the call 8 days from now (since tomorrow's call will be focused on the GitHub Secure Open Source Fund).

It would be great if this were a labextension too, so users have a way to specify these secrets in the UI.

[jtpio]

While I would stop short of proposing to make a new repo like jupyter-sercets, I wonder if shifting the code within the monorepo to a new package to create a "Key Manager" and making it a server extension with a way to pass keys as environment variables to kernels (and possible a UI for adding more secrets) could be in scope of jupyter-ai?

Looks like it would indeed make sense to have such functionality available outside of Jupyter AI to manage secrets in general. Maybe one of the following options?

1. in the JupyterLab monorepo directly, as a new package: https://github.com/jupyterlab/jupyterlab
2. as an extension under https://github.com/jupyterlab/jupyterlab

Option 2 could give the flexibility to iterate more quickly on it, before an eventual integration in core JupyterLab?

[krassowski]

I am just worried about the cost of maintaining yet another extension that would need to integrate with jupyter-ai, hence my suggestion to have it in this monorepo (same way as jupyter-collaboration has multiple packages). That said, if it was in JupyterLab core monorepo the maintenance cost would not be as high as if it was a standalone extension.

[jtpio]

Maybe it could be worth trying to develop it as a separate extension from the beginning, and see how it goes?

One of the concerns about having that in the Jupyter AI is that it is not really an AI-specific feature, even though the primary use case would be for storing API keys. There is also a risk that the extension is developed with only the Jupyter AI use case in mind (for example assuming a Jupyter Server extension), while there are already some projects out there that would benefit from a generic secrets manager not necessarily backed by a server extension:

• https://github.com/jupyterlite/ai
• https://github.com/QuantStack/jupydrive-s3
• https://github.com/jupyterlab/jupyterlab-github (for client side use)

If we have it in the monorepo in the beginning, there is a chance it will get extracted anyway later. This was the case already with Jupyter Chat which had to be extracted to a separate repo so it can be used more easily in more scenarios.

[BwL1289]

+1