Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Qunsong/qa eval groundedness service #34797

Merged
merged 17 commits into from
Mar 26, 2024
Merged

Qunsong/qa eval groundedness service #34797

merged 17 commits into from
Mar 26, 2024

Conversation

qusongms
Copy link
Contributor

@qusongms qusongms commented Mar 16, 2024 •
edited
Loading

Description

Update built-in QA evaluation flow:

  • remove prompt template based gpt_groundedness evaluation
  • add groundedness service
  • change harm level "Safe" to "Very low"

If an SDK is being regenerated based on a new swagger spec, a link to the pull request containing these swagger spec changes has been included above.

All SDK Contribution checklist:

  • [ x] The pull request does not introduce [breaking changes]
  • CHANGELOG is updated for new features, bug fixes or other significant changes.
  • I have read the contribution guidelines.

General Guidelines and Best Practices

  • Title of the pull request is clear and informative.
  • There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

  • Pull request includes test coverage for the included changes.

@qusongms
modify built-in qa evaluation flow
306c099 
* move groundedness evaluation to groundedness evaluation service
* change content harm level "Safe" to "Very low"
@qusongms
add default groundedness results to qa eval flow
e39550f
@github-actions github-actions bot added AI Community Contribution Community members are working on the issue labels Mar 16, 2024
@github-actions GitHub Actions
Copy link

Thank you for your contribution @qusongms! We will review the pull request and get back to you soon.

@github-actions github-actions bot added the customer-reported Issues that are reported by GitHub users external to the Azure organization. label Mar 16, 2024
singankit
singankit previously approved these changes Mar 25, 2024
View reviewed changes
@singankit singankit enabled auto-merge (squash) March 25, 2024 23:12
@qusongms
update built-in qa evaluation flow
fe8e197 
* fix fallback check logic in validate_groundedness_service
* add e2e test of fallback groundedness
auto-merge was automatically disabled March 26, 2024 02:18

Head branch was pushed to by a user without write access

@singankit singankit enabled auto-merge (squash) March 26, 2024 04:49
@github-actions GitHub Actions
Copy link

Hi @qusongms. We've noticed that new changes have been pushed to this pull request. Because it is set to automatically merge, we've reset the approvals to allow the opportunity to review the updates.

@github-actions github-actions bot dismissed singankit’s stale review March 26, 2024 05:48

Hi @singankit. We've noticed that new changes have been pushed to this pull request. Because it is set to automatically merge, we've reset the approvals to allow the opportunity to review the updates.

@github-actions GitHub Actions
Copy link

Hi @qusongms. We've noticed that new changes have been pushed to this pull request. Because it is set to automatically merge, we've reset the approvals to allow the opportunity to review the updates.

@github-actions GitHub Actions
Copy link

Hi @qusongms. We've noticed that new changes have been pushed to this pull request. Because it is set to automatically merge, we've reset the approvals to allow the opportunity to review the updates.

@github-actions GitHub Actions
Copy link

Hi @qusongms. We've noticed that new changes have been pushed to this pull request. Because it is set to automatically merge, we've reset the approvals to allow the opportunity to review the updates.

@github-actions GitHub Actions
Copy link

Hi @qusongms. We've noticed that new changes have been pushed to this pull request. Because it is set to automatically merge, we've reset the approvals to allow the opportunity to review the updates.

@github-actions GitHub Actions
Copy link

Hi @qusongms. We've noticed that new changes have been pushed to this pull request. Because it is set to automatically merge, we've reset the approvals to allow the opportunity to review the updates.

@singankit singankit merged commit 9973a20 into Azure:main Mar 26, 2024
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@singankit singankit singankit approved these changes

@Man-MSFT Man-MSFT Man-MSFT approved these changes

@luigiw luigiw Awaiting requested review from luigiw luigiw is a code owner

@needuv needuv Awaiting requested review from needuv needuv is a code owner

@paulshealy1 paulshealy1 Awaiting requested review from paulshealy1 paulshealy1 is a code owner

Assignees
No one assigned
Labels
AI Community Contribution Community members are working on the issue customer-reported Issues that are reported by GitHub users external to the Azure organization.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@qusongms @singankit @Man-MSFT