-
-
Notifications
You must be signed in to change notification settings - Fork 15
Create chat message attachments via drag-and-drop from files, notebook cells #248
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Updated demo video is in the PR description, now it shows current / latest state of this PR. |
41e5bfe
to
093a40b
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@andrii-i Thank you for building this! The new UX is awesome. Left some feedback below, mostly concerning type safety.
BTW:
Ability to drag text selection to create an attachment is not implemented in this PR as JupyterLab's drag events only carry plain text, no context about which cell/notebook they came from. Since Jupyter Chat attachments are references to specific locations in files we can't create meaningful attachments from dragged text selection as they are essentially plain text with unknown origin.
I checked on this and I couldn't find a reference for where text selections are handled by JupyterLab (searching for mimedata.setData()
). This may actually be native behavior from the browser, since text drags seems to be handled as "cut and paste" by default in browsers.
I think this is OK. Since modern LLM token limits are higher, users should attach the entire file/cell to maximize the context the LLM receives anyways. The persona may be expected to intelligently handle large files by running a grep
call first to see which lines in a large file are relevant.
If we're able to reach consensus that this behavior is OK, we should open a follow-up issue to remove the selection
fields from the IAttachment
models, since they will not be used.
I have been testing this locally, a few comments and questions:
|
Thank you for looking into this @ellisonbg.
Currently if multiple cells are dragged into the chat, single
Currently one could select all cells of a notebook and it would be attached as a single instance of
I agree with this, would prefer to follow up on this in a separate issue/PR as attachment previews are currently displayed via custom jupyter-chat/packages/jupyter-chat/src/components/attachments.tsx Lines 28 to 39 in f24e62f
Attachment paths are:
This follows JupyterLab's path conventions. |
What if you drag multiple cells from the same notebook, but at different
times (different drag/drop actions)? Looks like that creates multiple
notebook attachments.
…On Thu, Jul 3, 2025 at 10:46 AM Andrii Ieroshenko ***@***.***> wrote:
*andrii-i* left a comment (jupyterlab/jupyter-chat#248)
<#248 (comment)>
Thank you for looking into this @ellisonbg <https://github.com/ellisonbg>.
I love the ability to drag a cell. I also see you can attach multiple
cells from the same notebook, but they appear as separate attachments. In
the underlying data model, is is a single notebook file attachment with a
list of cell attachments, or multiple notebook file attachments each with a
single cell?
Currently if multiple cells are dragged into the chat, single
INotebookAttachment is created with cells added as INotebookAttachment.cells:
INotebookAttachmentCell[]. Here's a video and a screenshot of this
interaction:
image.png (view on web)
<https://github.com/user-attachments/assets/e3e0da9d-014d-4ecc-86dd-2661a6feff51>
dragndrop_mult_cells.gif (view on web)
<https://github.com/user-attachments/assets/91f07bfe-05d3-4a2e-97a5-856dade38268>
Eventually (can be in a separate PR) it would be nice to also drag entire
files that are already open, rather than just from the file browser.
Currently one could select all cells of a notebook and it would be
attached as a single instance of INotebookAttachment with multiple cells.
There is no way to drag entire non-notebook files that are already open,
would make sense if there would be. I'd say UI needs discussion so it would
make sense to follow up in a separate PR.
I see the "x" button to remove an attachment has a small click area. That
can also be a follow up issue/PR.
I agree with this, would prefer to follow up on this in a separate
issue/PR as attachment previews are currently displayed via custom
AttachmentPreviewList component
https://github.com/jupyterlab/jupyter-chat/blob/f24e62f9084780c2e2a76be4cde9838222604a1d/packages/jupyter-chat/src/components/attachments.tsx#L28-L39
similar to MUI Chips <https://mui.com/material-ui/react-chip/>. I'd say
that using MUI Chips and other pre-made components as much as possible
would improve usability, functionality, and UIUX overall.
What exact format are the paths in? Relative to server root? URL style
with forward slashes (even on Windows)? URL encoding?
Attachment paths are:
1. Paths are relative to ContentsManager.root_dir as documented in the
interfaces
2. Re forward slashes, JupyterLab uses POSIX-style paths internally
regardless of OS. Server Contents API takes care of that
3. No URL encoding, JupyterLab style POSIX paths are stored raw
This follows JupyterLab's path conventions.
—
Reply to this email directly, view it on GitHub
<#248 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAGXUHASPVRORNF7WRZRJT3GVUA7AVCNFSM6AAAAACANTHQTWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTAMZTGA3TGNJWGE>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
Brian E. Granger
Senior Principal Technologist, AWS AI/ML ***@***.***)
On Leave - Professor of Physics and Data Science, Cal Poly
@ellisonbg on GitHub
|
@ellisonbg that's correct, I haven't understood the question. Multiple separate drags would create multiple |
Separate PR is fine for that. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
minor non-blocking comments
Updated demo / video preview is in the PR description. |
I have tested this locally again. As long as the above code comments have been addressed, I think this is ready to go. @andrii-i can you resolve the comments above that have been addressed? Once those are clean I can merge. The UX of this is soo nice - can't wait to use it :-) |
Co-authored-by: David L. Qiu <david@qiu.dev>
Co-authored-by: David L. Qiu <david@qiu.dev>
Co-authored-by: David L. Qiu <david@qiu.dev>
Looks good, merging |
Problem
There is no way to quickly drag content from the file browser or notebook cells directly into the chat input area, users need to manually attach files and notebook cells to chat messages using the attach button.
Proposed solution
Use updated attachments API supporting cells and selection ranges introduced in #247 to allow users to drag content from the file browser or notebook cells directly into the chat input area to create attachments:
Testing instructions:
Not implemented
Ability to drag text selection to create an attachment is not implemented in this PR as JupyterLab's drag events only carry plain text, no context about which cell/notebook they came from. Since Jupyter Chat attachments are references to specific locations in files we can't create meaningful attachments from dragged text selection as they are essentially plain text with unknown origin.
This can be solved by enhancing JupyterLab's drag system upstream and implementing custom MIME type.