Implement streaming for `/fix` #1048

srdas · 2024-10-23T05:32:44Z

Fixes Implement response streaming in /fix #1041 by implementing response streaming in /fix

Implementation uses LCEL Runnables with streaming.

Video showing /fix streaming:
https://github.com/user-attachments/assets/99aa1eb2-b1a4-45dc-bd17-f0a20768198c

packages/jupyter-ai/jupyter_ai/chat_handlers/fix.py

dlqqq

@srdas Thanks for working on this! I've left a few comments below.

packages/jupyter-ai/jupyter_ai/chat_handlers/fix.py

dlqqq · 2024-10-28T22:51:21Z

Rebased & fixed merge conflicts.

srdas · 2024-10-28T23:32:21Z

@meeseeksdev please backport to v3-dev

Co-authored-by: Sanjiv Das <srdas@scu.edu>

* Backport PR #1049: Added new Anthropic Sonnet3.5 v2 models (#1050) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR #1051: Added Developer documentation for streaming responses (#1058) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR #1048: Implement streaming for `/fix` (#1059) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR #1057: [pre-commit.ci] pre-commit autoupdate (#1060) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Backport PR #1064: Added Ollama to the providers table in user docs (#1066) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR #1056: Add examples of using Fields and EnvAuthStrategy to developer documentation (#1073) Co-authored-by: Alan Meeson <alanmeeson@users.noreply.github.com> * Backport PR #1069: Merge Anthropic language model providers (#1076) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR #1068: Allow `$` to literally denote quantities of USD in chat (#1079) Co-authored-by: david qiu <david@qiu.dev> * Backport PR #1075: Fix magic commands when using non-chat providers w/ history (#1080) Co-authored-by: Alan Meeson <alanmeeson@users.noreply.github.com> * Backport PR #1077: Fix `/export` by including streamed agent messages (#1081) Co-authored-by: Mahmut CAVDAR <4072246+mcavdar@users.noreply.github.com> * Backport PR #1072: Reduced padding in cell around code icons in code toolbar (#1084) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR #1087: Improve installation documentation and clarify provider dependencies (#1091) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR #1092: Remove retired models and add new `Haiku-3.5` model in Anthropic (#1093) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR #1094: Continue to allow `$` symbols to delimit inline math in human messages (#1095) Co-authored-by: david qiu <david@qiu.dev> * Backport PR #1097: Update `faiss-cpu` version range (#1101) Co-authored-by: david qiu <david@qiu.dev> * Backport PR #1104: Fix rendering of code blocks in JupyterLab 4.3.0+ (#1105) Co-authored-by: david qiu <david@qiu.dev> * Backport PR #1106: Catch error on non plaintext files in `@file` and reply gracefully in chat (#1110) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR #1109: Bump LangChain minimum versions (#1112) Co-authored-by: david qiu <david@qiu.dev> * Backport PR #1119: Downgrade spurious 'error' logs (#1124) Co-authored-by: ctcjab <joshua.bronson@chicagotrading.com> * Backport PR #1127: Removes outdated OpenAI models and adds new ones (#1130) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR #1131: [pre-commit.ci] pre-commit autoupdate (#1132) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Backport PR #1125: Update model fields immediately on save (#1133) Co-authored-by: david qiu <david@qiu.dev> * Backport PR #1139: Fix install step in CI (#1140) Co-authored-by: david qiu <david@qiu.dev> * Backport PR #1129: Fix JSON serialization error in Ollama models (#1141) Co-authored-by: Mr.W <janus.choy@gmail.com> * Backport PR #1137: Update completion model fields immediately on save (#1142) Co-authored-by: david qiu <david@qiu.dev> * [v3-dev] Initial migration to `jupyterlab-chat` (#1043) * Very first version of the AI working in jupyterlab_collaborative_chat * Allows both collaborative and regular chat to work with AI * handle the help message in the chat too * Autocompletion (#2) * Fix handler methods' parameters * Add slash commands (autocompletion) to the chat input * Stream messages (#3) * Allow for stream messages * update jupyter collaborative chat dependency * AI settings (#4) * Add a menu option to open the AI settings * Remove the input option from the setting widget * pre-commit * linting * Homogeneize typing for optional arguments * Fix import * Showing that the bot is writing (answering) (#5) * Show that the bot is writing (answering) * Update jupyter chat dependency * Some typing * Update extension to jupyterlab_chat (0.6.0) (#8) * Fix linting * Remove try/except to import jupyterlab_chat (not optional anymore), and fix typing * linter * Python unit tests * Fix typing * lint * Fix lint and mypy all together * Fix web_app settings accessor * Fix jupyter_collaboration version Co-authored-by: david qiu <44106031+dlqqq@users.noreply.github.com> * Remove unecessary try/except * Dedicate one set of chat handlers per room (#9) * create new set of chat handlers per room * make YChat an instance attribute on BaseChatHandler * revert changes to chat handlers * pre-commit * use room_id local var Co-authored-by: Nicolas Brichet <32258950+brichet@users.noreply.github.com> --------- Co-authored-by: Nicolas Brichet <32258950+brichet@users.noreply.github.com> --------- Co-authored-by: david qiu <44106031+dlqqq@users.noreply.github.com> Co-authored-by: david qiu <david@qiu.dev> * Backport PR #1134: Improve user messaging and documentation for Cross-Region Inference on Amazon Bedrock (#1143) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Backport PR #1136: Add base API URL field for Ollama and OpenAI embedding models (#1149) Co-authored-by: Sanjiv Das <srdas@scu.edu> * [v3-dev] Remove `/export`, `/clear`, and `/fix` (#1148) * remove /export * remove /clear * remove /fix * Fix CI in `v3-dev` branch (#1154) * fix check release by bumping to impossible version * fix types * Update Playwright Snapshots --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * [v3-dev] Dedicate one LangChain history object per chat (#1151) * dedicate a separate LangChain history object per chat * pre-commit * fix mypy * Backport PR #1160: Trigger update snapshots based on commenter's role (#1161) Co-authored-by: david qiu <david@qiu.dev> * Backport PR #1155: Fix code output format in IPython (#1162) Co-authored-by: Divyansh Choudhary <divyanshchoudhary99@gmail.com> * Backport PR #1158: Update `/generate` to not split classes & functions across cells (#1164) Co-authored-by: Sanjiv Das <srdas@scu.edu> * Remove v2 frontend components (#1156) * First pass to remove the front end chat * Remove code-toolbar by using a simplified markdown renderer in settings * Remove chat-message-menu (should be ported in jupyter-chat) * Remove chat handler * Follow up 'Remove chat-message-menu (should be ported in jupyter-chat)' commit * Clean package.json * Remove UI tests * Remove the generative AI menu * Remove unused components * run yarn dedupe --------- Co-authored-by: David L. Qiu <david@qiu.dev> * Upgrade to `jupyterlab-chat>=0.7.0` (#1166) * upgrade to jupyterlab-chat 0.7.0 * pre-commit * upgrade to @jupyter/chat ^0.7.0 in frontend * Remove v2 backend components (#1168) * remove v2 llm memory, implement ReplyStream * remove v2 websockets & REST handlers * remove unused v2 data models * fix slash command autocomplete * fix unit tests * remove unused _learned context provider * fix mypy * pre-commit * fix optional k arg in YChatHistory * bump jupyter chat to 0.7.1 to fix Python 3.9 tests * revert accidentally breaking /learn --------- Co-authored-by: Lumberbot (aka Jack) <39504233+meeseeksmachine@users.noreply.github.com> Co-authored-by: Sanjiv Das <srdas@scu.edu> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Alan Meeson <alanmeeson@users.noreply.github.com> Co-authored-by: Mahmut CAVDAR <4072246+mcavdar@users.noreply.github.com> Co-authored-by: ctcjab <joshua.bronson@chicagotrading.com> Co-authored-by: Mr.W <janus.choy@gmail.com> Co-authored-by: Nicolas Brichet <32258950+brichet@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Divyansh Choudhary <divyanshchoudhary99@gmail.com>

srdas added the enhancement New feature or request label Oct 23, 2024

srdas changed the title ~~Adding developer documentation for streaming features and updated /fix and /ask to return streaming results~~ Updated /fix and /ask to return streaming results Oct 23, 2024

srdas changed the title ~~Updated /fix and /ask to return streaming results~~ Updated /fix to return streaming results Oct 23, 2024

srdas marked this pull request as ready for review October 23, 2024 17:28

srdas requested a review from dlqqq October 23, 2024 19:03

krassowski reviewed Oct 24, 2024

View reviewed changes

packages/jupyter-ai/jupyter_ai/chat_handlers/fix.py Show resolved Hide resolved

dlqqq mentioned this pull request Oct 24, 2024

Added Developer documentation for streaming responses #1051

Merged

dlqqq requested changes Oct 24, 2024

View reviewed changes

dlqqq requested changes Oct 25, 2024

View reviewed changes

packages/jupyter-ai/jupyter_ai/chat_handlers/fix.py Outdated Show resolved Hide resolved

packages/jupyter-ai/jupyter_ai/chat_handlers/fix.py Outdated Show resolved Hide resolved

dlqqq requested changes Oct 28, 2024

View reviewed changes

packages/jupyter-ai/jupyter_ai/chat_handlers/fix.py Outdated Show resolved Hide resolved

packages/jupyter-ai/jupyter_ai/chat_handlers/fix.py Outdated Show resolved Hide resolved

dlqqq changed the title ~~Updated /fix to return streaming results~~ Implement streaming for /fix Oct 28, 2024

dlqqq approved these changes Oct 28, 2024

View reviewed changes

dlqqq and others added 12 commits October 28, 2024 15:46

doc for streaming and added streaming for /fix

3424de1

removed additional docs

6c9c700

added custom pending message

1603a8a

Update fix.py

dd33f2b

Update fix.py

9fd1880

fall back

5bffec2

Update fix.py

fb90c79

Update base.py

fc12db5

Update fix.py

f5870ce

Update fix.py

18b288b

Update fix.py

c5fdaa1

add back anthropic models removed in rebase

63dff52

dlqqq force-pushed the streaming branch from 12838ac to 63dff52 Compare October 28, 2024 22:51

srdas merged commit dba7e4b into jupyterlab:main Oct 28, 2024
10 checks passed

meeseeksmachine pushed a commit to meeseeksmachine/jupyter-ai that referenced this pull request Oct 28, 2024

Backport PR jupyterlab#1048: Implement streaming for /fix

9d32a2c

meeseeksmachine mentioned this pull request Oct 28, 2024

Backport PR #1048 on branch v3-dev (Implement streaming for /fix) #1059

Merged

srdas added a commit that referenced this pull request Oct 28, 2024

Backport PR #1048: Implement streaming for /fix (#1059)

117a7be

Co-authored-by: Sanjiv Das <srdas@scu.edu>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement streaming for `/fix` #1048

Implement streaming for `/fix` #1048

srdas commented Oct 23, 2024 •

edited

Loading

dlqqq left a comment

dlqqq commented Oct 28, 2024

srdas commented Oct 28, 2024

Implement streaming for /fix #1048

Implement streaming for /fix #1048

Conversation

srdas commented Oct 23, 2024 • edited Loading

dlqqq left a comment

Choose a reason for hiding this comment

dlqqq commented Oct 28, 2024

srdas commented Oct 28, 2024

Implement streaming for `/fix` #1048

Implement streaming for `/fix` #1048

srdas commented Oct 23, 2024 •

edited

Loading