fix(evals)!: remove positional arguments in favor of kwargs #9996
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Enforces kwargs everywhere except the
.evaluateand.async_evaluatemethods onEvaluatorswhich only require one argument and have one optional argument.Also removes the evaluator registry which was not very useful and keeps implementation simpler.
Note
Enforces keyword-only parameters across the evals public API and removes the evaluator registry/
list_evaluators, updating code, tests, and tutorials accordingly.Evaluator.__init__,LLMEvaluator.__init__,ClassificationEvaluator.__init__,create_evaluator,create_classifier,bind_evaluator,evaluate_dataframe,async_evaluate_dataframe,utils.remap_eval_input,utils.to_annotation_dataframe.evaluate(eval_input=..., input_mapping=...))._registry) andlist_evaluators; drop exports from__init__.pyandevaluators.__all__.bind_evaluator,evaluate_dataframe,to_annotation_dataframe, etc.).Written by Cursor Bugbot for commit c9af39b. This will update automatically on new commits. Configure here.