Skip to content

Integrate inspect_ai for evaluation #22

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Jan 25, 2025
Prev Previous commit
Next Next commit
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
  • Loading branch information
pre-commit-ci[bot] committed Jan 24, 2025
commit b817134742ecf802beafd2d622d7600eedd438d3
4 changes: 2 additions & 2 deletions eval/inspect_ai/scicode.py
Original file line number Diff line number Diff line change
Expand Up @@ -245,9 +245,9 @@ def run_script(script_path):
subprocess.run(['python', script_path], check=True, capture_output=True,
text=True, timeout=1800)
return 0
except subprocess.CalledProcessError as e:
except subprocess.CalledProcessError:
return 1
except subprocess.TimeoutExpired as e:
except subprocess.TimeoutExpired:
return 2

total_steps = len(sub_steps)
Expand Down
Loading