Skip to content

Conversation

shyamnamboodiripad
Copy link
Contributor

@shyamnamboodiripad shyamnamboodiripad commented Feb 13, 2025

Microsoft Reviewers: Open in CodeFlow

@shyamnamboodiripad
Copy link
Contributor Author

@RussKie @stephentoub Looks like the dotnet-extensions-ai team is being included as required on all eval library PRs despite the following lines in the CODEOWNERS files (where dotnet-extensions-ai-evaluation appears ahead of dotnet-extensions-ai).

Any ideas on how we can fix the reviewer tagging to only include the dotnet-extensions-ai-evaluation team?

https://github.com/dotnet/extensions/blob/main/.github/CODEOWNERS#L20C1-L22C116

@shyamnamboodiripad shyamnamboodiripad enabled auto-merge (squash) February 13, 2025 20:44
@shyamnamboodiripad shyamnamboodiripad added the area-ai-eval Microsoft.Extensions.AI.Evaluation and related label Feb 13, 2025
@RussKie
Copy link
Contributor

RussKie commented Feb 13, 2025

@RussKie @stephentoub Looks like the dotnet-extensions-ai team is being included as required on all eval library PRs despite the following lines in the CODEOWNERS files (where dotnet-extensions-ai-evaluation appears ahead of dotnet-extensions-ai).

Any ideas on how we can fix the reviewer tagging to only include the dotnet-extensions-ai-evaluation team?

main/.github/CODEOWNERS#L20C1-L22C116

It's because of this:

/src/Libraries/Microsoft.Extensions.AI @dotnet/dotnet-extensions-ai
/src/Libraries/Microsoft.Extensions.AI.* @dotnet/dotnet-extensions-ai
/test/Libraries/Microsoft.Extensions.AI @dotnet/dotnet-extensions-ai
/test/Libraries/Microsoft.Extensions.AI.* @dotnet/dotnet-extensions-ai

It could be that the order is reversed. If reversing doesn't help, then you'll need to explicitly call out MEAI libraries.
https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners

@shyamnamboodiripad shyamnamboodiripad requested a review from a team as a code owner February 13, 2025 20:59
@shyamnamboodiripad
Copy link
Contributor Author

@RussKie @stephentoub Looks like the dotnet-extensions-ai team is being included as required on all eval library PRs despite the following lines in the CODEOWNERS files (where dotnet-extensions-ai-evaluation appears ahead of dotnet-extensions-ai).
Any ideas on how we can fix the reviewer tagging to only include the dotnet-extensions-ai-evaluation team?
main/.github/CODEOWNERS#L20C1-L22C116

It's because of this:

/src/Libraries/Microsoft.Extensions.AI @dotnet/dotnet-extensions-ai
/src/Libraries/Microsoft.Extensions.AI.* @dotnet/dotnet-extensions-ai
/test/Libraries/Microsoft.Extensions.AI @dotnet/dotnet-extensions-ai
/test/Libraries/Microsoft.Extensions.AI.* @dotnet/dotnet-extensions-ai

It could be that the order is reversed. If reversing doesn't help, then you'll need to explicitly call out MEAI libraries. https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners

@RussKie Thanks - I have pushed a commit to this PR that reverses the order. Could you please approve if it looks good?

@shyamnamboodiripad
Copy link
Contributor Author

shyamnamboodiripad commented Feb 13, 2025

FYI @peterwald Looks like CI flaked out with the following error (same as #5901)

VSTest(0,0): error : (NETCORE_ENGINEERING_TELEMETRY=Build) Tests failed: /mnt/vss/_work/1/s/artifacts/TestResults/Release/Microsoft.Extensions.Caching.Hybrid.Tests_net8.0_x64.html [net8.0|x64]

Looks like only the Ubuntu leg failed - Windows is past the point of failure - so perhaps the failure is intermittent / non-deterministic.

@dotnet-comment-bot
Copy link
Collaborator

‼️ Found issues ‼️

Project Coverage Type Expected Actual
Microsoft.Extensions.Caching.Hybrid Line 86 82.77 🔻
Microsoft.Extensions.AI.Ollama Line 80 78.2 🔻
Microsoft.Extensions.AI.Evaluation.Quality Line 88 7.57 🔻
Microsoft.Extensions.AI.Evaluation.Quality Branch 88 16.42 🔻
Microsoft.Extensions.AI.Evaluation.Console Line 88 8.26 🔻
Microsoft.Extensions.AI.Evaluation.Console Branch 88 17.07 🔻
Microsoft.Extensions.AI.Evaluation Line 88 58.67 🔻
Microsoft.Extensions.AI.Evaluation Branch 88 56.67 🔻
Microsoft.Gen.MetadataExtractor Line 98 57.35 🔻
Microsoft.Gen.MetadataExtractor Branch 98 62.5 🔻
Microsoft.Extensions.AI.OpenAI Line 77 68.19 🔻
Microsoft.Extensions.AI.OpenAI Branch 77 50.2 🔻
Microsoft.Extensions.AI.Evaluation.Reporting Line 88 72.06 🔻
Microsoft.Extensions.AI.Evaluation.Reporting Branch 88 64.8 🔻

🎉 Good job! The coverage increased 🎉
Update MinCodeCoverage in the project files.

Project Expected Actual
Microsoft.Extensions.AI 88 89
Microsoft.Extensions.AI.AzureAIInference 91 92

Full code coverage report: https://dev.azure.com/dnceng-public/public/_build/results?buildId=951837&view=codecoverage-tab

@dotnet-comment-bot
Copy link
Collaborator

‼️ Found issues ‼️

Project Coverage Type Expected Actual
Microsoft.Extensions.AI.Evaluation.Quality Line 88 7.57 🔻
Microsoft.Extensions.AI.Evaluation.Quality Branch 88 16.42 🔻
Microsoft.Extensions.AI.Evaluation.Console Line 88 8.26 🔻
Microsoft.Extensions.AI.Evaluation.Console Branch 88 17.07 🔻
Microsoft.Extensions.Caching.Hybrid Line 86 82.77 🔻
Microsoft.Extensions.AI.Evaluation Line 88 58.67 🔻
Microsoft.Extensions.AI.Evaluation Branch 88 56.67 🔻
Microsoft.Extensions.AI.Ollama Line 80 78.2 🔻
Microsoft.Extensions.AI.OpenAI Line 77 68.04 🔻
Microsoft.Extensions.AI.OpenAI Branch 77 49.8 🔻
Microsoft.Extensions.AI.Evaluation.Reporting Line 88 72.06 🔻
Microsoft.Extensions.AI.Evaluation.Reporting Branch 88 64.8 🔻
Microsoft.Gen.MetadataExtractor Line 98 57.35 🔻
Microsoft.Gen.MetadataExtractor Branch 98 62.5 🔻

🎉 Good job! The coverage increased 🎉
Update MinCodeCoverage in the project files.

Project Expected Actual
Microsoft.Extensions.AI.AzureAIInference 91 92
Microsoft.Extensions.AI 88 89

Full code coverage report: https://dev.azure.com/dnceng-public/public/_build/results?buildId=951897&view=codecoverage-tab

@shyamnamboodiripad shyamnamboodiripad merged commit c1ea357 into dotnet:main Feb 13, 2025
6 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators Mar 16, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-ai-eval Microsoft.Extensions.AI.Evaluation and related
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants