Implement the board game Hive and its expansions

[rhstephens]

This is my implementation of Hive. While I plan on continually making improvements, what's here is in a good working state and has been tested thoroughly for correctness. I hope that the matter of copyright gets sorted out at some point, but for now, I figured I'd at least put up what I have for review!

I copied the README.md I wrote down below, as it goes into most of the implementation details in depth:

Hive

Implements the base game of Hive and its three expansion pieces: Mosquito, Ladybug, and Pillbug.

Example game state viewed on the command-line (left) and with an external viewer "Mzinga" (right)

This implementation follows the rules outlined by the Universal Hive Protocol (UHP), which means states can be serialized to and deserialized from valid UHP game strings. With a bit of I/O handling, this can also be used as a UHP-compliant Hive Engine, making interactions with other engines straightforward.

State

Observation Tensor

First, the hexagonal grid needs to be represented as a rectangular one for 2D convolution:

Example transformation - taken from RedBlobGames

The observation tensor then takes the form of multiple 2D feature planes describing the board and turn state, similar to what was done for AlphaZero chess.

However, since Hive's "board" is a repeating hexagonal tiling, the size is bounded only by the maximum number of tiles that can be laid in a straight line (28 total tiles for all expansions). Yet, a grid of size 28x28 is far too large to be computationally practical.

To help offset the complications this would bring for training in AlphaZero, the board can be paramaterized with board_size to reduce the tensor's overall sparsity. Using a board_size smaller than kMaxBoardSize means that some outlier games cannot be perfectly represented and are instead forced to a Draw. In practice, games that would approach that board length are extremely rare, so the trade-off feels acceptable.

The 2D feature planes are one-hot encodings that indicate:

• the presence of a particular bug type, for each player
• which bugs are pinned
• which bugs are covered
• the available positions that each player can place a new bug tile
• all 1's or all 0's to distinguish the current player's turn

Action Space

An action in Hive is described as:

1. choosing which tile to move
2. choosing which tile it moves next to (or on top of)
3. the relative direction of the tile it moves next to

e.g. "wA2 bL/" - White moves their 2nd Ant to the top right edge of Black's Ladybug

With there being 28 unique tiles and 7 directions (the 6 hexagonal edges and "above"), the action space can be thought of as entries into a 3D matrix with dimensions 7 x 28 x 28 = 5488 total actions.

This is not a perfect action space representation as there are a handful of unused actions (e.g. moving a tile next to itself?), but it does capture every legal move. Unfortunately, with the introduction of the Pillbug, each player is able to move their own piece or the enemy's, meaning we can't implicitly expect the tile being moved to be the colour of the current player. This ends up doubling the action space size from 7x14x28 to 7x28x28

To-do

Below are some concrete features and fixes I intend to implement to either help speed up training or improve the interoperability between other Hive software (e.g. displaying games directly to MzingaViewer):

• Address the efficiency of code that uses the most compute time (HiveState::GenerateValidSlides() and HiveState::IsGated() from recent perf tests)
• Implement zobrist hashing to handle a "3-repeated moves" forced draw (unofficial community rule)
• Undo()
• Perft()
• Make it possible to load many UHP gamestrings from a file for training, or to collect interesting game statistics
• Create a separate binary that handles I/O and behaves as a proper UHP-compliant engine
• Provide a simplified action space for games that do not use expansion pieces

Future Improvements / Thoughts

While developing this engine, I came across many interesting ideas that have the potential for serious progress towards a viable expert-level AZ-bot for Hive. And as of this submission, no such Hive AI exists, making the prospect of any improvements much more appealing.

Below is a record of those miscellaneous thoughts, in approximate order of the potential I think it has:

• Design a more exact action space. There are a handful of other suggested notations from the Hive community, each with their own advantages and drawbacks, that may be useful to look into for an alternative action space. One that looks very promising is Direction-Based Notation, as it implicitly covers all rotations and reflections by design.

• Use a Hexagonal CNN model or filter. One problem that has been conveniently unaddressed is the fact that 2D convolution is performed on Hexagonal data that has be refitted onto a square. The typical 3x3 filter then doesn't accurately describe the 6 neighbours of a hex, as 2 extra values are contained in the filter. One option would be to use a custom 3x3 filter that zeroes-out the two values along the diagonal, or to attempt using a more advanced implementation like HexCNN or Rotational-Invariant CNN. The first option would be much easier to implement into the existing AlphaZero framework.

• Attempt a graph/node-based representation. With how a game of Hive is structed like a graph itself, I think there is potential in using Graph Neural Networks (GNN) for learning. Some recent research has been done by applying GNNs to AlphaZero for board game AI, which indicates there is at least some proven success already.

[lanctot]

@rhstephens This looks awesome!!

Just chiming in quickly to say sorry for the delay in response. I was on holiday and have been catching up a putting out a few fires since my return. I'll take a proper look soon.

[lanctot]

Meanwhile, could I ask that you pull changes from master and push the merge commit? We needed to fix a few things to get Github Actions CI working, so you will need changes from #1250.

Once that's done, I'll approve and run the tests and take a deeper look after that.

[rhstephens]

No worries at all, thanks for responding! I've rebased my changes on top of #1250, but let me know if anything else needs to be done on my end.

Also, if needed, I'm more than willing to go over any parts of the code that may need clarification. Just let me know

[lanctot]

Is this explained somewhere?

Actually, better yet: can you make this value (5488) a constant at the top of the file and put the explanation near it?

constexpr int kNumDistinctActions = 5488;

and then return it here.

(btw the number of actions doesn't change with any expansions?)

[rhstephens]

I had something in the readme, but not in code. There are 28 unique tiles and 7 directions a tile can be placed beside (the 6 hexagonal edges and "above"). So the total action space is tile from * tile to * num directions == 28 * 28 * 7 == 5488.

I agree, this value should be a constant and with a comment explaining it like above. I'll make that change

As for it staying as 5488 regardless of expansion... I made this choice for a few reasons, mainly that mixing and matching expansions isn't really relevant to the Hive community which almost exclusively plays with all 3 expansions. But also, trying to account for any possible expansion combination would make the code a lot more confusing and messier without being able to rely on the BugType and HiveTile::Value Enums for iteration and indexing, among many other issues. It just didn't seem worth the change when a removal of one expansion bug only reduces the action space from 5488 to 5103

[lanctot]

Happy to keep the action space constant across the expansions (simpler).

[lanctot]

Ok then I think this is good to important once you make that change.

[lanctot]

A few minor comments for now (I may add more), but on the first glance the code looks very nice. Well done!