Skip to content

create button to stop streaming of response #1884

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

create button to stop streaming of response #1884

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
71dae75
create button to stop streaming of response
ssjdan27 Aug 4, 2024
6141939
Merge branch 'main' into main
ssjdan27 Aug 4, 2024
987ebed
Merge branch 'main' into ssjdan27/main
pamelafox Aug 31, 2024
ac7a78b
Update ja translation
pamelafox Sep 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions app/frontend/src/api/api.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ export async function askApi(request: ChatAppRequest, idToken: string | undefine
return parsedResponse as ChatAppResponse;
}

export async function chatApi(request: ChatAppRequest, shouldStream: boolean, idToken: string | undefined): Promise<Response> {
export async function chatApi(request: ChatAppRequest, shouldStream: boolean, idToken: string | undefined, signal: AbortSignal): Promise<Response> {
let url = `${BACKEND_URI}/chat`;
if (shouldStream) {
url += "/stream";
Expand All @@ -50,7 +50,8 @@ export async function chatApi(request: ChatAppRequest, shouldStream: boolean, id
return await fetch(url, {
method: "POST",
headers: { ...headers, "Content-Type": "application/json" },
body: JSON.stringify(request)
body: JSON.stringify(request),
signal: signal
});
}

Expand Down
23 changes: 18 additions & 5 deletions app/frontend/src/components/QuestionInput/QuestionInput.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { useState, useEffect, useContext } from "react";
import { Stack, TextField } from "@fluentui/react";
import { Button, Tooltip } from "@fluentui/react-components";
import { Send28Filled } from "@fluentui/react-icons";
import { Send28Filled, Stop24Filled } from "@fluentui/react-icons";
import { useTranslation } from "react-i18next";

import styles from "./QuestionInput.module.css";
Expand All @@ -16,9 +16,11 @@ interface Props {
placeholder?: string;
clearOnSend?: boolean;
showSpeechInput?: boolean;
onStop?: () => void;
isStreaming: boolean;
}

export const QuestionInput = ({ onSend, disabled, placeholder, clearOnSend, initQuestion, showSpeechInput }: Props) => {
export const QuestionInput = ({ onSend, onStop, disabled, placeholder, clearOnSend, initQuestion, showSpeechInput, isStreaming }: Props) => {
const [question, setQuestion] = useState<string>("");
const { loggedIn } = useContext(LoginContext);
const { t } = useTranslation();
Expand Down Expand Up @@ -87,9 +89,20 @@ export const QuestionInput = ({ onSend, disabled, placeholder, clearOnSend, init
onCompositionEnd={handleCompositionEnd}
/>
<div className={styles.questionInputButtonsContainer}>
<Tooltip content={t("tooltips.submitQuestion")} relationship="label">
<Button size="large" icon={<Send28Filled primaryFill="rgba(115, 118, 225, 1)" />} disabled={sendQuestionDisabled} onClick={sendQuestion} />
</Tooltip>
{isStreaming ? (
<Tooltip content={t("tooltips.stopStreaming")} relationship="label">
<Button size="large" icon={<Stop24Filled primaryFill="rgba(255, 0, 0, 1)" />} onClick={onStop} />
</Tooltip>
) : (
<Tooltip content={t("tooltips.submitQuestion")} relationship="label">
<Button
size="large"
icon={<Send28Filled primaryFill="rgba(115, 118, 225, 1)" />}
disabled={sendQuestionDisabled}
onClick={sendQuestion}
/>
</Tooltip>
)}
</div>
{showSpeechInput && <SpeechInput updateQuestion={setQuestion} />}
</Stack>
Expand Down
1 change: 1 addition & 0 deletions app/frontend/src/locales/en/translation.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@

"tooltips": {
"submitQuestion": "Submit question",
"stopStreaming": "Stop streaming",
"askWithVoice": "Ask question with voice",
"stopRecording": "Stop recording question",
"showThoughtProcess": "Show thought process",
Expand Down
1 change: 1 addition & 0 deletions app/frontend/src/locales/es/translation.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@

"tooltips": {
"submitQuestion": "Enviar pregunta",
"stopStreaming": "Detener la transmisión",
"askWithVoice": "Realizar pregunta con voz",
"stopRecording": "Detener la grabación de la pregunta",
"showThoughtProcess": "Mostrar proceso de pensamiento",
Expand Down
1 change: 1 addition & 0 deletions app/frontend/src/locales/fr/translation.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@

"tooltips": {
"submitQuestion": "Soumettre une question",
"stopStreaming": "Arrêter la diffusion",
"askWithVoice": "Poser une question à l'aide de la voix",
"stopRecording": "Arrêter l'enregistrement de la question",
"showThoughtProcess": "Montrer le processus de réflexion",
Expand Down
1 change: 1 addition & 0 deletions app/frontend/src/locales/ja/translation.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@

"tooltips":{
"submitQuestion": "質問を送信",
"stopStreaming": "ストリーミングを停止",
"askWithVoice": "音声で質問",
"stopRecording": "質問の記録を停止",
"showThoughtProcess": "思考プロセスの表示",
Expand Down
1 change: 1 addition & 0 deletions app/frontend/src/pages/ask/Ask.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -259,6 +259,7 @@ export function Component(): JSX.Element {
initQuestion={question}
onSend={question => makeApiRequest(question)}
showSpeechInput={showSpeechInput}
isStreaming={false}
/>
</div>
</div>
Expand Down
57 changes: 54 additions & 3 deletions app/frontend/src/pages/chat/Chat.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,8 @@ const Chat = () => {

const [isLoading, setIsLoading] = useState<boolean>(false);
const [isStreaming, setIsStreaming] = useState<boolean>(false);
const [partialResponse, setPartialResponse] = useState<string>("");
const [abortController, setAbortController] = useState<AbortController | null>(null);
const [error, setError] = useState<unknown>();

const [activeCitation, setActiveCitation] = useState<string>();
Expand Down Expand Up @@ -108,14 +110,15 @@ const Chat = () => {
});
};

const handleAsyncRequest = async (question: string, answers: [string, ChatAppResponse][], responseBody: ReadableStream<any>) => {
const handleAsyncRequest = async (question: string, answers: [string, ChatAppResponse][], responseBody: ReadableStream<any>, signal: AbortSignal) => {
let answer: string = "";
let askResponse: ChatAppResponse = {} as ChatAppResponse;

const updateState = (newContent: string) => {
return new Promise(resolve => {
setTimeout(() => {
answer += newContent;
setPartialResponse(answer);
const latestResponse: ChatAppResponse = {
...askResponse,
message: { content: answer, role: askResponse.message.role }
Expand All @@ -128,6 +131,9 @@ const Chat = () => {
try {
setIsStreaming(true);
for await (const event of readNDJSONStream(responseBody)) {
if (signal.aborted) {
break;
}
if (event["context"] && event["context"]["data_points"]) {
event["message"] = event["delta"];
askResponse = event as ChatAppResponse;
Expand All @@ -141,8 +147,11 @@ const Chat = () => {
throw Error(event["error"]);
}
}
} catch (e) {
console.error("error in handleAsyncRequest: ", e);
} finally {
setIsStreaming(false);
setPartialResponse("");
}
const fullResponse: ChatAppResponse = {
...askResponse,
Expand All @@ -155,6 +164,8 @@ const Chat = () => {
const { loggedIn } = useContext(LoginContext);

const makeApiRequest = async (question: string) => {
const controller = new AbortController();
setAbortController(controller);
lastQuestionRef.current = question;

error && setError(undefined);
Expand Down Expand Up @@ -197,15 +208,15 @@ const Chat = () => {
session_state: answers.length ? answers[answers.length - 1][1].session_state : null
};

const response = await chatApi(request, shouldStream, token);
const response = await chatApi(request, shouldStream, token, controller.signal);
if (!response.body) {
throw Error("No response body");
}
if (response.status > 299 || !response.ok) {
throw Error(`Request failed with status ${response.status}`);
}
if (shouldStream) {
const parsedResponse: ChatAppResponse = await handleAsyncRequest(question, answers, response.body);
const parsedResponse: ChatAppResponse = await handleAsyncRequest(question, answers, response.body, controller.signal);
setAnswers([...answers, [question, parsedResponse]]);
} else {
const parsedResponse: ChatAppResponseOrError = await response.json();
Expand All @@ -232,6 +243,7 @@ const Chat = () => {
setStreamedAnswers([]);
setIsLoading(false);
setIsStreaming(false);
setPartialResponse("");
};

useEffect(() => chatMessageStreamEnd.current?.scrollIntoView({ behavior: "smooth" }), [isLoading]);
Expand Down Expand Up @@ -317,6 +329,16 @@ const Chat = () => {
setSelectedAnswer(index);
};

const onStopClick = async () => {
try {
if (abortController) {
abortController.abort();
}
} catch (e) {
console.log("An error occurred trying to stop the stream: ", e);
}
};

// IDs for form labels and their associated callouts
const promptTemplateId = useId("promptTemplate");
const promptTemplateFieldId = useId("promptTemplateField");
Expand Down Expand Up @@ -393,6 +415,33 @@ const Chat = () => {
</div>
</div>
))}
{partialResponse && !isStreaming && (
<div>
<UserChatMessage message={lastQuestionRef.current} />
<div className={styles.chatMessageGpt}>
<Answer
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is it that we need to have this if/else with a whole Answer component configuration, versus sending in a parameter to Answer?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good question. I believe I remember running into weird issues when dealing with the partial responses that would sometimes cause the last messages to disappear. This was put into place to circumvent that, but I am all ears if there is a cleaner implementation!

isStreaming={false}
answer={{
message: { content: partialResponse, role: "assistant" },
delta: { content: partialResponse, role: "assistant" },
context: {
data_points: [],
followup_questions: null,
thoughts: []
},
session_state: undefined
}}
isSelected={false}
onCitationClicked={() => {}}
onThoughtProcessClicked={() => {}}
onSupportingContentClicked={() => {}}
onFollowupQuestionClicked={() => {}}
showFollowupQuestions={false}
speechUrl={null}
/>
</div>
</div>
)}
{!isStreaming &&
answers.map((answer, index) => (
<div key={index}>
Expand Down Expand Up @@ -443,6 +492,8 @@ const Chat = () => {
disabled={isLoading}
onSend={question => makeApiRequest(question)}
showSpeechInput={showSpeechInput}
isStreaming={isStreaming}
onStop={onStopClick}
/>
</div>
</div>
Expand Down