Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor Stepwise Planner handler, added logic to allow use of stepwise plan result as bot response #514

Merged
merged 3 commits into from
Oct 18, 2023

Conversation

teresaqhoang
Copy link
Contributor

@teresaqhoang teresaqhoang commented Oct 17, 2023

Motivation and Context

This pull request refactors the use of Stepwise Planner in CC. The changes include a fix to embed the result of the Stepwise Planner into the meta prompt if a valid result is returned and adds a new property to PlannerOptions to indicate whether to use the planner result as the bot response.

Additionally, the ChatSkill class has been updated to handle the new changes, including adding the ability to pass a response string to the HandleBotResponseAsync method and changing the StepwiseThoughtProcess property to a PlanExecutionMetadata object. The PromptDialog component has also been updated to display the Stepwise Planner supplement and raw view when appropriate, and the BotResponsePrompt interface has been updated to include a rawView property (to show the Stepwise Thought Process if the webapi is configured to return the stepwise response directly).

Description

Logic:

  1. If Stepwise Planner doesn't return a valid result (i.e., "Result not found"), continue with regular response generation. Planner result will not be included in meta prompt.
    image

  2. If Stepwise Planner returns a valid result, that result will be supplemented with complementary text to guide the model in using this response in the meta prompt.
    image

  3. If Stepwise Planner returns a valid result and the webapi is configured to use this result directly, it will be immediately returned to the client.
    image

Change Details:

  • Update ExternalInformation property in BotResponsePrompt to use SemanticDependency of PlanExecutionMetadata instead of ISemanticDependency.
  • Rename StepwiseThoughtProcess class to PlanExecutionMetadata.
  • Add RawResult property to PlanExecutionMetadata to store the raw result of the planner.
  • Add UseStepwiseResultAsBotResponse property to PlannerOptions to indicate whether to use the planner result as the bot response.
  • Add StepwisePlannerSupplement property to PromptsOptions to help guide model in using a response from StepwisePlanner.
  • Update ChatSkill to use SemanticDependency of PlanExecutionMetadata instead of StepwiseThoughtProcess.
  • Add logic to handle using plan result as bot response if UseStepwiseResultAsBotResponse is true.
  • Update HandleBotResponseAsync method to accept rawResult parameter.

Stepwise Thought Proceses available in Prompt details anytime the Stepwise planner is used, even if no result is found.
image

Contribution Checklist

@teresaqhoang teresaqhoang requested review from TaoChenOSU, dehoward and a team October 17, 2023 16:24
@teresaqhoang teresaqhoang self-assigned this Oct 17, 2023
@github-actions github-actions bot added webapp Pull requests that update Typescript code webapi Pull requests that update .net code labels Oct 17, 2023
@teresaqhoang teresaqhoang added this pull request to the merge queue Oct 18, 2023
Merged via the queue into microsoft:main with commit 8924709 Oct 18, 2023
8 checks passed
@teresaqhoang teresaqhoang deleted the stepwise-planner-refactor branch October 18, 2023 17:03
Ryangr0 pushed a commit to webgrip/chat-copilot that referenced this pull request Oct 26, 2023
…se plan result as bot response (microsoft#514)

### Motivation and Context

<!-- Thank you for your contribution to the chat-copilot repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

This pull request refactors the use of Stepwise Planner in CC. The
changes include a fix to embed the result of the Stepwise Planner into
the meta prompt if a valid result is returned and adds a new property to
PlannerOptions to indicate whether to use the planner result as the bot
response.

Additionally, the ChatSkill class has been updated to handle the new
changes, including adding the ability to pass a response string to the
HandleBotResponseAsync method and changing the StepwiseThoughtProcess
property to a PlanExecutionMetadata object. The PromptDialog component
has also been updated to display the Stepwise Planner supplement and raw
view when appropriate, and the BotResponsePrompt interface has been
updated to include a rawView property (to show the Stepwise Thought
Process if the webapi is configured to return the stepwise response
directly).

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

Logic:
1. If Stepwise Planner doesn't return a valid result (i.e., "Result not
found"), continue with regular response generation. Planner result will
not be included in meta prompt.

![image](https://github.com/microsoft/chat-copilot/assets/125500434/d25168b4-3bb2-4148-ad3a-1ded6d93fe75)

2. If Stepwise Planner returns a valid result, that result will be
supplemented with complementary text to guide the model in using this
response in the meta prompt.

![image](https://github.com/microsoft/chat-copilot/assets/125500434/61622a4c-61f4-4b10-8af0-36d9aed014b2)

3. If Stepwise Planner returns a valid result and the webapi is
configured to use this result directly, it will be immediately returned
to the client.

![image](https://github.com/microsoft/chat-copilot/assets/125500434/8f4cdf8e-67e7-4fde-9bcb-e2fdc42fbc3b)

Change Details:
- Update ExternalInformation property in BotResponsePrompt to use
SemanticDependency of PlanExecutionMetadata instead of
ISemanticDependency.
- Rename StepwiseThoughtProcess class to PlanExecutionMetadata.
- Add RawResult property to PlanExecutionMetadata to store the raw
result of the planner.
- Add UseStepwiseResultAsBotResponse property to PlannerOptions to
indicate whether to use the planner result as the bot response.
- Add StepwisePlannerSupplement property to PromptsOptions to help guide
model in using a response from StepwisePlanner.
- Update ChatSkill to use SemanticDependency of PlanExecutionMetadata
instead of StepwiseThoughtProcess.
- Add logic to handle using plan result as bot response if
UseStepwiseResultAsBotResponse is true.
- Update HandleBotResponseAsync method to accept rawResult parameter.

Stepwise Thought Proceses available in Prompt details anytime the
Stepwise planner is used, even if no result is found.

![image](https://github.com/microsoft/chat-copilot/assets/125500434/6b14dc70-f79e-47be-a0b3-0833c9b14117)


### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [Contribution
Guidelines](https://github.com/microsoft/chat-copilot/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/chat-copilot/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
~- [ ] All unit tests pass, and I have added new tests where possible~
- [x] I didn't break anyone 😄
teamleader-dev pushed a commit to vlink-group/chat-copilot that referenced this pull request Oct 7, 2024
…se plan result as bot response (microsoft#514)

### Motivation and Context

<!-- Thank you for your contribution to the chat-copilot repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

This pull request refactors the use of Stepwise Planner in CC. The
changes include a fix to embed the result of the Stepwise Planner into
the meta prompt if a valid result is returned and adds a new property to
PlannerOptions to indicate whether to use the planner result as the bot
response.

Additionally, the ChatSkill class has been updated to handle the new
changes, including adding the ability to pass a response string to the
HandleBotResponseAsync method and changing the StepwiseThoughtProcess
property to a PlanExecutionMetadata object. The PromptDialog component
has also been updated to display the Stepwise Planner supplement and raw
view when appropriate, and the BotResponsePrompt interface has been
updated to include a rawView property (to show the Stepwise Thought
Process if the webapi is configured to return the stepwise response
directly).

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

Logic:
1. If Stepwise Planner doesn't return a valid result (i.e., "Result not
found"), continue with regular response generation. Planner result will
not be included in meta prompt.

![image](https://github.com/microsoft/chat-copilot/assets/125500434/d25168b4-3bb2-4148-ad3a-1ded6d93fe75)

2. If Stepwise Planner returns a valid result, that result will be
supplemented with complementary text to guide the model in using this
response in the meta prompt.

![image](https://github.com/microsoft/chat-copilot/assets/125500434/61622a4c-61f4-4b10-8af0-36d9aed014b2)

3. If Stepwise Planner returns a valid result and the webapi is
configured to use this result directly, it will be immediately returned
to the client.

![image](https://github.com/microsoft/chat-copilot/assets/125500434/8f4cdf8e-67e7-4fde-9bcb-e2fdc42fbc3b)

Change Details:
- Update ExternalInformation property in BotResponsePrompt to use
SemanticDependency of PlanExecutionMetadata instead of
ISemanticDependency.
- Rename StepwiseThoughtProcess class to PlanExecutionMetadata.
- Add RawResult property to PlanExecutionMetadata to store the raw
result of the planner.
- Add UseStepwiseResultAsBotResponse property to PlannerOptions to
indicate whether to use the planner result as the bot response.
- Add StepwisePlannerSupplement property to PromptsOptions to help guide
model in using a response from StepwisePlanner.
- Update ChatSkill to use SemanticDependency of PlanExecutionMetadata
instead of StepwiseThoughtProcess.
- Add logic to handle using plan result as bot response if
UseStepwiseResultAsBotResponse is true.
- Update HandleBotResponseAsync method to accept rawResult parameter.

Stepwise Thought Proceses available in Prompt details anytime the
Stepwise planner is used, even if no result is found.

![image](https://github.com/microsoft/chat-copilot/assets/125500434/6b14dc70-f79e-47be-a0b3-0833c9b14117)


### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [Contribution
Guidelines](https://github.com/microsoft/chat-copilot/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/chat-copilot/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
~- [ ] All unit tests pass, and I have added new tests where possible~
- [x] I didn't break anyone 😄
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
webapi Pull requests that update .net code webapp Pull requests that update Typescript code
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

3 participants