Evaluation Pipelines broken: Gradio, E2B

[kanishkg]

I have been trying to evaluate samples from a model and all the evaluation methods seem to not be working.

Gradio

Command to replicate: bigcodebench.evaluate --model meta-llama/Meta-Llama-3.1-8B-Instruct --execution gradio --split instruct --subset hard --backend vllm
Gradio was working yesterday, but shows an error now.
Error:

ValueError: Could not fetch config for https://bigcode-bigcodebench-evaluator.hf.space/

E2B

Command to replicate: bigcodebench.evaluate --model meta-llama/Meta-Llama-3.1-8B-Instruct --execution e2b --split instruct --subset hard --backend vllm
E2B shows the following error:

Traceback (most recent call last):
  File "/scr/kanishkg/miniconda3/envs/cesos/bin/bigcodebench.evaluate", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/scr/kanishkg/miniconda3/envs/cesos/lib/python3.11/site-packages/bigcodebench/evaluate.py", line 432, in main
    Fire(evaluate)
  File "/scr/kanishkg/miniconda3/envs/cesos/lib/python3.11/site-packages/fire/core.py", line 135, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/scr/kanishkg/miniconda3/envs/cesos/lib/python3.11/site-packages/fire/core.py", line 468, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
                                ^^^^^^^^^^^^^^^^^^^^
  File "/scr/kanishkg/miniconda3/envs/cesos/lib/python3.11/site-packages/fire/core.py", line 684, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
                ^^^^^^^^^^^^^^^^^^^^^^
  File "/scr/kanishkg/miniconda3/envs/cesos/lib/python3.11/site-packages/bigcodebench/evaluate.py", line 192, in evaluate
    sandbox.commands.run("bigcodebench.evaluate  --execution 'local' "
  File "/scr/kanishkg/miniconda3/envs/cesos/lib/python3.11/site-packages/e2b/sandbox_sync/commands/command.py", line 212, in run
    else proc.wait(
         ^^^^^^^^^^
  File "/scr/kanishkg/miniconda3/envs/cesos/lib/python3.11/site-packages/e2b/sandbox_sync/commands/command_handle.py", line 133, in wait
    raise CommandExitException(
e2b.sandbox.commands.command_handle.CommandExitException: Command exited with code 2 and error:
/bin/bash: -c: line 1: syntax error near unexpected token `('

[terryyz]

I believe that Hugging Face was having some network issues, so the space was dead when it was restarted.
Just a note that you can duplicate this space and change the Gradio endpoint to your own in case there are some issues with the main space.

Regarding E2B, working on it now.

[terryyz]

fixed