Expressivity-of-Emergent-Languages

The official code base of Expressivity of Emergent Language is a Trade-off between Contextual Complexity and Unpredictability (ICLR-2022).

Table of Contents

1. Citation Information
2. Installation
3. Run Experiments
4. Plot Result
5. License

---

Citation Information

The citation information of our paper is given as follows:

@inproceedings{
guo2022expressivity,
title={Expressivity of Emergent Languages is a Trade-off between Contextual Complexity and Unpredictability},
author={Shangmin Guo and Yi Ren and Kory Wallace Mathewson and Simon Kirby and Stefano V Albrecht and Kenny Smith},
booktitle={International Conference on Learning Representations},
year={2022},
url={https://openreview.net/forum?id=WxuE_JWxjkW}
}

---

Installation

To install the necessary packages for running our codes, you can use the bash commands provided below.

1. Clone the repo.

git clone git@github.com:uoe-agents/Expressivity-of-Emergent-Languages.git;
cd Expressivity-of-Emergent-Languages

2. Create a virtual environment for the repo.

python3 -m venv expressivity;
source expressivity/bin/activate

3. Install the editdistance package for EGG_research module.

pip install wheel editdistance;

4. Install lib/EGG_research.

cd lib/EGG_research;
pip install .

5. Install other dependencies.

cd ../../
pip install -r requirements.txt;

To verify whether the installation succeeds, you can run a demo game by:

bash ./bashes/run_demo_games.sh

---

Run Experiments

Prediction 1 and Prediction 3

In order to verify Prediction 1 and Prediction 3 in our paper, you can run the bash script context_num.sh in the directory bashes, by e.g.

bash ./bashes/context_num.sh

Note that, you can preset random seeds by editing the seed list in context_num.sh. For example, the first two lines can be updated to

# list random seeds below
for s in 404 65535

Prediction 2

To verify Prediction 2 in our paper, you can run the bash script same_com_diff_pred.sh in the directory bashes, by e.g.

bash ./bashes/same_com_diff_pred.sh

Note that, similar to the above, you can preset random seeds by editing the seed list in same_com_diff_pred.sh. For example, the first two lines can be updated to

# list random seeds below
for s in 404 65535

---

Plot Results

Once the experiments have finished, the diagram we provided in our paper can be easily generated by:

python context_analysis.py --log-path ./log_to_your_exp1 # default value is ./log/
python scdp_analysis_analysis.py --log-path ./log_to_your_exp2 # default value is ./log_scdp/

The default path for collecting the diagrams is results, you can also specify another path by setting --result-path in the above two commands.

The most important three diagrams are:

1. generalisation_vs_complexity.pdf to verify Prediction 1, and it should look similar to the following figure.

2. diffs_statistics_on_all_sources.pdf to verify Prediction 2, and it should look similar to the following figure.

3. acc_target_vs_source.pdf to verify Prediction 3, , and it should look similar to the following figure.

---

License

All the source code was licensed (and remains so) under the GNU General Public License v3.0 (included in LICENSE file). Any new code is also licensed under the GNU General Public License v3.0.