Skip to content

Commit 173bb18

Browse files
committed
fix(llmobs): fix assessment argument description for custom evals (#15032)
Corrects the docstring description of the `assessment` argument in `submit_evaluation()`. There is no functional change but the docstring description was incorrect. <!-- Provide an overview of the change and motivation for the change --> <!-- Describe your testing strategy or note what tests are included --> <!-- Note any risks associated with this change, or "None" if no risks --> <!-- Any other information that would be helpful for reviewers --> (cherry picked from commit 314c76e)
1 parent 4735346 commit 173bb18

File tree

3 files changed

+8
-3
lines changed

3 files changed

+8
-3
lines changed

ddtrace/llmobs/_llmobs.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1630,7 +1630,7 @@ def submit_evaluation(
16301630
If not set, the current time will be used.
16311631
:param dict metadata: A JSON serializable dictionary of key-value metadata pairs relevant to the
16321632
evaluation metric.
1633-
:param str assessment: An assessment of the validity of this evaluation. Must be either "pass" or "fail".
1633+
:param str assessment: An assessment of this evaluation. Must be either "pass" or "fail".
16341634
:param str reasoning: An explanation of the evaluation result.
16351635
"""
16361636
if span_context is not None:
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
fixes:
3+
- |
4+
LLM Observability: Corrected the description of the ``assessment`` argument in ``submit_evaluation()``.
5+
``assessment`` now refers to whether the evaluation itself passes or fails according to your application, rather than the validity of the evaluation result.

tests/llmobs/test_llmobs_service.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1879,7 +1879,7 @@ def test_submit_evaluation_enqueues_writer_with_assessment(llmobs, mock_llmobs_e
18791879
tags={"foo": "bar", "bee": "baz", "ml_app": "ml_app_override"},
18801880
ml_app="ml_app_override",
18811881
metadata={"foo": ["bar", "baz"]},
1882-
assessment="pass",
1882+
assessment="fail",
18831883
)
18841884
mock_llmobs_eval_metric_writer.enqueue.assert_called_with(
18851885
_expected_llmobs_eval_metric_event(
@@ -1891,7 +1891,7 @@ def test_submit_evaluation_enqueues_writer_with_assessment(llmobs, mock_llmobs_e
18911891
categorical_value="high",
18921892
tags=["ddtrace.version:{}".format(ddtrace.__version__), "ml_app:ml_app_override", "foo:bar", "bee:baz"],
18931893
metadata={"foo": ["bar", "baz"]},
1894-
assessment="pass",
1894+
assessment="fail",
18951895
)
18961896
)
18971897
mock_llmobs_eval_metric_writer.reset()

0 commit comments

Comments
 (0)