Can intelligent agents play Minecraft? Developing a hierarchical reinforcement learning agent to play Minecraft.

MineRL Markov Model:

Method

To find out how subtask state transitions are transferred, we implemented a Markov Model to analyze them. We first split the whole IronPickaxe dataset into different subtasks, e.g.: Start, Log, Planks, Crafting tables, etc, to extract the subtasks. After that, to keep consistency, we filtered some poor-quality trajectories. Finally, the most greedy subtask path was generated by our Markov model.

Transition Heatmap for Obtain IronPickaxe task:

Visualization of the Markov model:

The most greedy subtask path:

['Start', 'log', 'planks', 'crafting_table', 'stick', 'wooden_pickaxe', 'cobblestone', 'stone_pickaxe', 'iron_ore', 'furnace', 'iron_ingot', 'iron_pickaxe', 'Termination']

MineRL HCI assistant

1. Subtask recommendation

 a. Assigning subtask for human-player.
 b. Subtask could be the chain in HDQfD or Markov-controller or hand-crafted chain.
 c. Potential research: Comparing Human-player performance between different provided chains.

2. Action recommendation

 a. Assigning action in the subtask for human players.
 b. For action, things might be a bit complex.
 c. 10 basic actions: 
    Move -- [forward, back, left, right]  
    Camera -- [up, down, left, right] 
    Combo -- [forward+jump] 
    Attack 
 d. Special actions:
    Craft -- [Planke, stick, crafting_table, wooden_pickaxe, etc]: converting raw material
    Place -- [item]: placing item(s)
    NearbyCraft -- [item]: Craft item(s) when something nearby
    Equip -- [item]

MineRL intelligent agent:

The MineRL intelligent agent is based on HDQfD (forgER) algorithm, see [Paper] [Github].
Notes:

• We cannot reproduce the results in the paper, mostly because of the MineRL environment updates and MineRL dataset issues.
  
• We changed some parts of the algorithm to improve the performance (see forger folder).