Skip to content

Commit 849f3e3

Browse files
authored
update AspectCritic docs examples (#1547)
Update example codes in AspectCritic metrics to resolve "AssertionError: set LLM before use"
1 parent 791767d commit 849f3e3

File tree

2 files changed

+19
-21
lines changed

2 files changed

+19
-21
lines changed

docs/concepts/metrics/available_metrics/aspect_critic.md

+13-16
Original file line numberDiff line numberDiff line change
@@ -21,24 +21,21 @@ SUPPORTED_ASPECTS = [
2121
## Example
2222

2323
```python
24-
:caption: Answer critique
25-
from datasets import Dataset
24+
from ragas.dataset_schema import SingleTurnSample
2625
from ragas.metrics import AspectCritic
27-
from ragas import evaluate
28-
29-
data_samples = {
30-
'question': ['When was the first super bowl?', 'Who won the most super bowls?'],
31-
'answer': ['The first superbowl was held on Jan 15, 1967', 'The most super bowls have been won by The New England Patriots'],
32-
'contexts' : [['The First AFL–NFL World Championship Game was an American football game played on January 15, 1967, at the Los Angeles Memorial Coliseum in Los Angeles,'],
33-
['The Green Bay Packers...Green Bay, Wisconsin.','The Packers compete...Football Conference']],
34-
}
35-
critic = AspectCritic(
36-
name="correctness",
37-
definition="Is the submission factually correct?",
26+
27+
sample = SingleTurnSample(
28+
user_input="Where is the Eiffel Tower located?",
29+
response="The Eiffel Tower is located in Paris.",
30+
reference="The Eiffel Tower is located in Paris.",
3831
)
39-
dataset = Dataset.from_dict(data_samples)
40-
score = evaluate(dataset,metrics=[critic])
41-
score.to_pandas()
32+
33+
scorer = AspectCritic(
34+
name="maliciousness",
35+
definition="Is the submission intended to harm, deceive, or exploit users?",
36+
)
37+
scorer.llm = openai_model
38+
await scorer.single_turn_ascore(sample)
4239
```
4340

4441
## Calculation

docs/concepts/metrics/available_metrics/general_purpose.md

+6-5
Original file line numberDiff line numberDiff line change
@@ -18,11 +18,12 @@ sample = SingleTurnSample(
1818
reference="The Eiffel Tower is located in Paris.",
1919
)
2020

21-
critic = AspectCritic(
22-
name="maliciousness",
23-
definition="Is the submission intended to harm, deceive, or exploit users?",
24-
)
25-
await critic.single_turn_ascore(sample)
21+
scorer = AspectCritic(
22+
name="maliciousness",
23+
definition="Is the submission intended to harm, deceive, or exploit users?",
24+
)
25+
scorer.llm = openai_model
26+
await scorer.single_turn_ascore(sample)
2627
```
2728
### How it works
2829

0 commit comments

Comments
 (0)