Skip to content

Conversation

@ehutt
Copy link
Contributor

@ehutt ehutt commented Oct 24, 2025

Enforces kwargs everywhere except the .evaluate and .async_evaluate methods on Evaluators which only require one argument and have one optional argument.

Also removes the evaluator registry which was not very useful and keeps implementation simpler.


Note

Enforces keyword-only parameters across the evals public API and removes the evaluator registry/list_evaluators, updating code, tests, and tutorials accordingly.

  • API changes (keyword-only):
    • Enforce kwargs in Evaluator.__init__, LLMEvaluator.__init__, ClassificationEvaluator.__init__, create_evaluator, create_classifier, bind_evaluator, evaluate_dataframe, async_evaluate_dataframe, utils.remap_eval_input, utils.to_annotation_dataframe.
    • Update internal and caller sites to pass named args (e.g., evaluate(eval_input=..., input_mapping=...)).
  • Removals:
    • Delete evaluator registry (_registry) and list_evaluators; drop exports from __init__.py and evaluators.__all__.
  • Tests & tutorials:
    • Refactor tests and notebooks to use the new keyword-only API; remove registry tests and adjust examples (bind_evaluator, evaluate_dataframe, to_annotation_dataframe, etc.).

Written by Cursor Bugbot for commit c9af39b. This will update automatically on new commits. Configure here.

@ehutt ehutt requested review from a team as code owners October 24, 2025 00:16
@github-project-automation github-project-automation bot moved this to 📘 Todo in phoenix Oct 24, 2025
@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Oct 24, 2025
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.6"
"version": "3.9.23"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Nonexistent Python Version in Notebook Metadata

Invalid Python version "3.9.23" in Jupyter notebook metadata. Python version 3.9.23 doesn't exist - the latest Python 3.9.x version is 3.9.21. This appears to be accidentally committed notebook metadata that could cause issues if tools try to validate or use this version number.

Fix in Cursor Fix in Web

@ehutt ehutt marked this pull request as draft October 24, 2025 16:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L This PR changes 100-499 lines, ignoring generated files.

Projects

Status: 📘 Todo

Development

Successfully merging this pull request may close these issues.

1 participant