Predator-Prey-Grass

Coevolution in a multi-agent deep reinforcement learning gridworld

This repo explores the interplay between nature (inherited traits via reproduction and mutation) and nurture (behavior learned via reinforcement learning) in ecological systems. We combine Multi-Agent Deep Reinforcement Learning (MADRL) with evolutionary dynamics to explore emergent behaviors in a multi-agent dynamic ecosystem of Predators, Prey, and regenerating Grass. Agents differ by speed, vision, energy metabolism, and decision policies—offering ground for open-ended adaptation. At its core lies a gridworld simulation where agents are not just trained—they are born, age, reproduce, die, and even mutate in a continuously changing environment.

Trained Predator-Prey-Grass walls-occlusion environment

Environments:

• Base environment: The two-policy base environment. (implementation, results)

• Mutating agents: A four-policy extension of the base environment. (implementation, results)

• Centralized training: A single-policy variant of the base environment (implementation)

• Walls occlusion: An extension with walls and occluded vision (implementation)

• Reproduction kick back rewards: On top of direct reproduction rewards, agents receive indirect rewards when their children reproduce (implementation)

• Lineage rewards: On top of direct reproduction rewards, agents receive rewards when their offspring survives over time (implementation)

Experiments:

• Testing the Red Queen Hypothesis in the co-evolutionary setting of (non-mutating) predators and prey (implementation, results)

• Testing the Red Queen Hypothesis in the co-evolutionary setting of mutating predators and prey (implementation, results)

Hyperparameter tuning

• Hyperparameter tuning base environment - Population-Based Training (Implementation)

Installation of the repository

Editor used: Visual Studio Code 1.106.3 on Linux Mint 22.0 Cinnamon

1. Clone the repository:

   git clone https://github.com/doesburg11/PredPreyGrass.git

2. Open Visual Studio Code and execute:
   • Press ctrl+shift+p
   • Type and choose: "Python: Create Environment..."
   • Choose environment: Conda
   • Choose interpreter: Python 3.11.13 or higher
   • Open a new terminal

   • pip install -e .

3. Install the additional system dependency for Pygame visualization:

   • conda install -y -c conda-forge gcc=14.2.0

Quick start

Run the pre-trained policy in a Visual Studio Code terminal:

python ./src/predpreygrass/rllib/base_environment/evaluate_ppo_from_checkpoint_debug.py

Or a random policy:

python ./src/predpreygrass/rllib/base_environment/random_policy.py

References

• RLlib: Industry-Grade, Scalable Reinforcement Learning
• Paper Collection of Multi-Agent Reinforcement Learning (MARL)
• Multi-Agent Reinforcement Learning: Foundations and Modern Approaches. Stefano V. Albrecht, Filippos Christianos, and Lukas Schäfer