Remove extra validation checks and add some tests for detect and evaluate #59
Conversation
tests/test_analyze.py
Outdated
| assert hasattr(response, 'status') or hasattr(response, 'hallucination') | ||
| assert any("deprecated" in str(warn.message).lower() for warn in w) | ||
|
|
||
| def test_analyze_eval_missing_header_error(self): |
There was a problem hiding this comment.
This test is the only one failing in the test suite. It's testing AnalyzeEval (the obsolete interface we still support). The SDK backend returns response 'task_definition must be a string' https://github.com/aimonlabs/aimon-sdk-backend/blob/d5aa4c8cb31b7ff3a79676dad74547c92195bc03/aimon-sdk-backend/service.py#L175. Not sure why this happens, task_definition is absent from query altogether and not needed, so the backend must not have checked if it's a string.
There was a problem hiding this comment.
yes lets remove AnalyzeEval since its been deprecated for a while
| "evaluation_id": am_eval.id, | ||
| "evaluation_run_id": eval_run.id, | ||
| } | ||
| if "prompt" in record and record["prompt"]: |
There was a problem hiding this comment.
when async is True, we still need to handle the validation but we can do that in the backend service
There was a problem hiding this comment.
looks like we have this handled in the backend 👍
|
@alexlyzhov lets bump the version in the setup.py |
aimon/_version.py
Outdated
| @@ -1,4 +1,4 @@ | |||
| # File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details. | |||
|
|
|||
| __title__ = "aimon" | |||
| __version__ = "0.9.2" | |||
| __version__ = "0.9.3" | |||
There was a problem hiding this comment.
lets do 0.10.0 since we are including the new IA detector
setup.py
Outdated
| @@ -8,7 +8,7 @@ | |||
| name='aimon', | |||
| python_requires='>3.8.0', | |||
| packages=find_packages(), | |||
| version="0.9.2", | |||
| version="0.9.3", | |||
Uh oh!
There was an error while loading. Please reload this page.