This directory archives Fast Downward experiments.

Experiments using Downward Lab
==============================

All experiments use their own common_setup.py module that simplifies the
creation of experiments. We recommend using common_setup.py (and related
*.py files) from issue891 for new experiments since it is the most
recent version.

To isolate the Downward Lab versions used for different experiments and
papers, we recommend using Python virtual environments. To create and
activate a virtual environment for an issue experiment, use the
following steps:

    unset PYTHONPATH  # If PYTHONPATH is set, unset it for a clean environment.
    cd issue891
    python3 -m venv --prompt=issue891 .venv
    source .venv/bin/activate  # enter the virtual environment
    # list exact versions of your dependencies in a file (for example, lab==4.2)
    pip install -r requirements.txt  # install dependencies
    ./v1-opt.py 1 2 3 4  # run your experiment
    deactivate  # leave the virtual environment

Below we list some experiments that show how certain tasks can be
performed. Please note that some of these experiments use outdated Lab and
Python versions.

- Test changes that affect the whole planner:
  - issue481/v1-*.py show the general setup. You will need different
    experiments for satisficing, optimal and anytime configurations.

- Add a custom log parser:
  - issue735/v1.py

- Compare all attributes for tasks where we lose coverage:
  - issue439/regressions.py

- Use custom time limit:
  - issue439/issue439.py

- Add scatter plots for custom attributes:
  - issue214/issue214.py

- Run configurations on the same tasks multiple times to reduce noise:
  - issue420/issue420-v1-regressions.py

- Independent CompareRevisionReports for portfolio configs and core solvers:
  - issue462/issue462-opt.py

- RelativeScatterPlotReport:
  - issue77/issue77-v7-opt.py

- M&S: configs and additional parsing:
  - issue914/v1.py and issue914/ms-parser.py

- Run experiment in debug mode (with assertions):
  - issue650/v2.py

- Translator:
  - issue862/v5.py is a translator-only experiment, including a report
    that only shows results where the translator output changed
  - issue887/v1.py is essentially the same, but with a before/after
    comparison rather than a "plain" multiple-configuration report.
  - issue862/v5-planner.py is a follow-up experiment to v5.py to see
    the impact of the translator changes on the overall planner; it is
    essentially a standard planner experiment though, not very
    translator-specific; the subset of domains to evaluate is
    specified manually


Microbenchmarks
===============

Some experiments don't run the whole planner or planner components, but
just contain small microbenchmarks for particular functionality. These
may be a good starting point for similar microbenchmarks. Examples:

- Benchmarking of random number generation:
  - issue269/rng-microbenchmark

If you add your own microbenchmark, it is recommended to start from a
copy of an existing example and follow the naming convention
issue[...]/[...]-microbenchmark for the code. This way, .hgignore
should be set up correctly out of the box.