---

Overview

MineStudio contains a series of tools and APIs that can help you quickly develop Minecraft AI agents:

• Simulator: Easily customizable Minecraft simulator based on MineRL.
• Data: A trajectory data structure for efficiently storing and retrieving arbitray trajectory segment.
• Models: A template for Minecraft policy model and a gallery of baseline models.
• Offline Training: A straightforward pipeline for pre-training Minecraft agents with offline data.
• Online Training: Efficient RL implementation supporting memory-based policies and simulator crash recovery.
• Inference: Pallarelized and distributed inference framework based on Ray.
• Benchmark: Automating and batch-testing of diverse Minecraft tasks.

This repository is under development. We welcome any contributions and suggestions.

Installation

For a more detailed installation guide, please refer to the documentation.

MineStudio requires Python 3.10 or later. We recommend using conda to maintain an environment on Linux systems. JDK 8 is also required for running the Minecraft simulator.

conda create -n minestudio python=3.10 -y
conda activate minestudio
conda install --channel=conda-forge openjdk=8 -y

MineStudio is available on PyPI. You can install it via pip.

pip install MineStudio

To install MineStudio from source, you can run the following command:

pip install git+https://github.com/CraftJarvis/MineStudio.git

Minecraft simulator requires rendering tools. For users with nvidia graphics cards, we recommend installing VirtualGL. For other users, we recommend using Xvfb, which supports CPU rendering but is relatively slower. Refer to the documentation for installation commands.

After the installation, you can run the following command to check if the installation is successful:

python -m minestudio.simulator.entry # using Xvfb
MINESTUDIO_GPU_RENDER=1 python -m minestudio.simulator.entry # using VirtualGL

Docker

We provide a Docker image for users who want to run MineStudio in a container. The Dockerfile is available in the assets directory. You can build and run the image by running the following command:

cd assets
docker build --platform=linux/amd64 -t minestudio .
docker run -it minestudio

Datasets on 🤗 Hugging Face

We converted the Contractor Data the OpenAI VPT project provided to our trajectory structure and released them to the Hugging Face.

• CraftJarvis/minestudio-data-6xx
• CraftJarvis/minestudio-data-7xx
• CraftJarvis/minestudio-data-8xx
• CraftJarvis/minestudio-data-9xx
• CraftJarvis/minestudio-data-10xx

from minestudio.data import load_dataset

dataset = load_dataset(
    mode='raw', 
    dataset_dirs=['6xx', '7xx', '8xx', '9xx', '10xx'], 
    enable_video=True,
    enable_action=True,
    frame_width=224, 
    frame_height=224,
    win_len=128, 
    split='train', 
    split_ratio=0.9, 
    verbose=True
)
item = dataset[0]
print(item.keys())

Models on 🤗 Hugging Face

We have pushed all the checkpoints to 🤗 Hugging Face, it is convenient to load the policy model.

from minestudio.simulator import MinecraftSim
from minestudio.simulator.callbacks import RecordCallback
from minestudio.models import VPTPolicy

policy = VPTPolicy.from_pretrained("CraftJarvis/MineStudio_VPT.rl_from_early_game_2x").to("cuda")
policy.eval()

env = MinecraftSim(
    obs_size=(128, 128), 
    callbacks=[RecordCallback(record_path="./output", fps=30, frame_type="pov")]
)
memory = None
obs, info = env.reset()
for i in range(1200):
    action, memory = policy.get_action(obs, memory, input_shape='*')
    obs, reward, terminated, truncated, info = env.step(action)
env.close()

Here is the checkpoint list:

• CraftJarvis/MineStudio_VPT.foundation_model_1x, trained by OpenAI
• CraftJarvis/MineStudio_VPT.foundation_model_2x, trained by OpenAI
• CraftJarvis/MineStudio_VPT.foundation_model_3x, trained by OpenAI
• CraftJarvis/MineStudio_VPT.bc_early_game_2x, trained by OpenAI
• CraftJarvis/MineStudio_VPT.rl_from_house_2x, trained by OpenAI
• CraftJarvis/MineStudio_VPT.rl_from_early_game_2x, trained by OpenAI
• CraftJarvis/MineStudio_VPT.bc_house_3x, trained by OpenAI
• CraftJarvis/MineStudio_VPT.bc_early_game_3x, trained by OpenAI
• CraftJarvis/MineStudio_VPT.rl_for_shoot_animals_2x, trained by CraftJarvis
• CraftJarvis/MineStudio_VPT.rl_for_build_portal_2x, trained by CraftJarvis
• CraftJarvis/MineStudio_STEVE-1.official, trained by STEVE-1
• CraftJarvis/MineStudio_ROCKET-1.12w_EMA, trained by CraftJarvis

Why MineStudio

Acknowledgement

The simulation environment is built upon MineRL and Project Malmo. We also refer to Ray, PyTorch Lightning for distributed training and inference. Thanks for their great work.

Citation

@inproceedings{MineStudio,
  title={MineStudio: A Streamlined Package for Minecraft AI Agent Development},
  author={Shaofei Cai and Zhancun Mu and Kaichen He and Bowei Zhang and Xinyue Zheng and Anji Liu and Yitao Liang},
  year={2024},
  url={https://api.semanticscholar.org/CorpusID:274992448}
}