Environments

Environments

Built-in (24)

Ship with pip install deepgym. Load by name:

from deepgym import load_environment
env = load_environment('coin_change')

Coding

Easy

Name	Family	Task
fizzbuzz	String	Classic FizzBuzz
reverse_string	String	Reverse a string
palindrome_check	String	Check if string is a palindrome
anagram_check	String	Check if two strings are anagrams
valid_parentheses	String	Validate bracket matching
python_sorting	Array	Sort an array
string_manipulation	String	String transform operations
two_sum	Hash Map	Find two numbers that sum to target

Medium

Name	Family	Task
coin_change	DP	Minimum coins to make amount
climbing_stairs	DP	Count ways to climb n stairs
house_robber	DP	Max loot without robbing adjacent houses
rotate_array	Array	Rotate array by k positions
remove_duplicates	Array	Remove duplicates from sorted array
max_subarray	Array	Maximum subarray sum (Kadane's)
roman_to_integer	String	Convert Roman numeral to integer
matrix_spiral	Array	Traverse matrix in spiral order
longest_consecutive	Hash Map	Longest consecutive element sequence
group_anagrams	Hash Map	Group strings that are anagrams
top_k_frequent	Hash Map	Find k most frequent elements
merge_intervals	Array	Merge overlapping intervals
binary_search	Search	Binary search implementation

Hard

Name	Family	Task
longest_common_subsequence	DP	LCS of two sequences
level_order_traversal	Tree	BFS tree traversal

Computer-use

Name	Task
file_organizer	Organize files using CLI/GUI commands
cli_task	Interact with a command-line tool

Tool-use

Name	Task
api_request	Make API calls and process responses
data_pipeline	Process data through a multi-step pipeline

---

Listing environments

from deepgym import list_environments

for env_info in list_environments():
    print(f"{env_info['name']:30s} {env_info['difficulty']:8s} {env_info['domain']}")

Suites

from deepgym import load_suite

easy = load_suite('easy')       # 8 environments
medium = load_suite('medium')   # 13 environments
hard = load_suite('hard')       # 2 environments

coding = load_suite('coding')
dp_envs = load_suite('dynamic-programming')
all_envs = load_suite('all')    # 24 environments

---

Importable benchmarks (2,350+)

Download standard coding benchmarks and convert them to DeepGym environments:

python scripts/import_humaneval.py      # 164 problems
python scripts/import_mbpp.py           # 500 problems
python scripts/import_evalplus.py       # HumanEval+ (80x more tests) + MBPP+
python scripts/import_bigcodebench.py   # 1,140 problems

After import, they land in ~/.deepgym/environments/ and you can load them like any other env:

env = load_environment('humaneval/HumanEval_0')
env = load_environment('mbpp/mbpp_1')
env = load_environment('bigcodebench/BigCodeBench_0')

---

Custom environments

Inline verifier

from deepgym import Environment

env = Environment(
    task='Write a function `add(a, b)` that returns the sum of two numbers.',
    verifier_code=(
        'import importlib.util, sys\n'
        'spec = importlib.util.spec_from_file_location("sol", solution_path)\n'
        'mod = importlib.util.module_from_spec(spec)\n'
        'spec.loader.exec_module(mod)\n'
        'return 1.0 if hasattr(mod, "add") and mod.add(2, 3) == 5 else 0.0\n'
    ),
)

File-based verifier

from pathlib import Path

env = Environment(
    task='Implement binary search.',
    verifier_path=Path('my_verifier.py'),
    timeout=60,
    difficulty='medium',
    tags=['search', 'algorithm'],
)

With test cases

env = Environment(
    task='Write `multiply(a, b)` that returns the product.',
    verifier_code=(
        'import importlib.util, json, sys\n'
        'spec = importlib.util.spec_from_file_location("sol", solution_path)\n'
        'mod = importlib.util.module_from_spec(spec)\n'
        'spec.loader.exec_module(mod)\n'
        'with open(test_cases_path) as f:\n'
        '    cases = json.load(f)\n'
        'passed = sum(1 for c in cases if mod.multiply(*c["input"]) == c["expected"])\n'
        'return passed / len(cases)\n'
    ),
    test_cases=[
        {'input': [2, 3], 'expected': 6},
        {'input': [0, 5], 'expected': 0},
        {'input': [-1, 7], 'expected': -7},
    ],
)

See Verifier Protocol for the full spec on writing verifiers.