Skip to content

Commit d1b04c0

Browse files
committed
add docs for custom eval metric
1 parent 0fb018a commit d1b04c0

File tree

1 file changed

+18
-0
lines changed

1 file changed

+18
-0
lines changed

pages/logging/log_via_api.mdx

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -114,6 +114,24 @@ Note:- A prompt run cannot have duplicate attribute names
114114
}
115115
```
116116

117+
##### Custom Eval Metrics
118+
119+
Optionally, you can also log custom eval metrics with your prompt. You can pass metric name and metric value as key-value pair in the `custom_eval_metrics` object.
120+
121+
Note:- A prompt run cannot have duplicate eval metric names
122+
123+
```json
124+
{
125+
// ...otherFields,
126+
"custom_eval_metrics": {
127+
"throughput": 100,
128+
"correctness": 0.5,
129+
"latency": 100,
130+
//any other eval metric to be logged
131+
}
132+
}
133+
```
134+
117135
##### Grounded Evals
118136

119137
For grounded evals like [Answer Similarity](/evals/preset_evals/grounded_evals#answer_similarity), you must also log a reference response (string) to compare against:

0 commit comments

Comments
 (0)