Added Increased Singularity/Apptainer support Benchmarking #4
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request introduces significant updates to the
InteractiveAgentTester.pyscript, expanding its functionality to support an additional backend (singularity-exec) and improving code readability and maintainability. The changes also include updates to the Singularity sandbox configuration file to accommodate the new backend.Backend enhancements:
singularity-execbackend, which enables offline REPL execution without networking. This includes a new_SingExecBackendclass to manage container lifecycle, data binding, and code execution.COPY_CMDandEXECUTE_ENDPOINTto handle backend-specific behavior. [1] [2]Code execution improvements:
run_interactivemethod to handle code execution differently forsingularity-execby using theexec_codemethod of_SingExecBackend. Networked backends continue to use HTTP requests.format_execute_responsemethod to process execution results consistently across backends, including handling images and output streams.Code readability and maintainability:
lsttoresincollect_resources. [1] [2]extract_python_codeanddisplayfor clarity and modularity. [1] [2]Singularity sandbox updates:
offline_kernel.pyto the Singularity sandbox definition file to support thesingularity-execbackend. This ensures the required REPL script is available inside the container.These changes collectively enhance the flexibility of the testing framework, allowing it to operate in environments with restricted networking while maintaining modular and clean code.