You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
New formating example.
```
Given a input and response. Evaluate the submission only using the given criteria. Use only 'Yes' (1) and 'No' (0) as verdict.
Please return the output in a JSON format that complies with the following schema as specified in JSON Schema:
{'properties': {'reason': {'description': 'Reason for the verdict', 'title': 'Reason', 'type': 'string'}, 'verdict': {'description': 'The verdict (0 or 1) for the submission', 'title': 'Verdict', 'type': 'integer'}}, 'required': ['reason', 'verdict'], 'title': 'AspectCriticOutput', 'type': 'object'}
--------EXAMPLES-----------
Example 1
Input: {
"user_input": "Who was the director of Los Alamos Laboratory?",
"response": "Einstein was the director of Los Alamos Laboratory.",
"criteria": "Is the output written in perfect grammar"
}
Output: {
"reason": "the criteria for evaluation is whether the output is written in perfect grammar. In this case, the output is grammatically correct.",
"verdict": 1
}
-----------------------------
Now perform the same with the following input
Input: {
"user_input": "What is x",
"response": "Y",
"criteria": "no one"
}
Output:
```
0 commit comments