[tune] HyperOpt AssertionError if setting `points_to_evaluate`

What is the problem?

The script at the bottom fails with AssertionError:

Exception has occurred: AssertionError
  File "..,/site-packages/hyperopt/pyll/base.py", line 904, in rec_eval
    assert aa is not GarbageCollected

This error doesn't occur if I don't set points_to_evaluate.

Ray version and other system information (Python version, TensorFlow version, OS):

ray: 2.0.0.dev0
hyperopt: 0.2.5

Reproduction (REQUIRED)

Please provide a short code snippet (less than 50 lines if possible) that can be copy-pasted to reproduce the issue. The snippet should have no external library dependencies (i.e., use fake or mock data / environments):

import ray
from ray import tune
from ray.tune.suggest.hyperopt import HyperOptSearch


if __name__ == "__main__":
    ray.init(num_cpus=2, local_mode=True)

    config = {
        "env": "CartPole-v1",

        "seed": tune.lograndint(1, int(1e9)),
        "model": {
            "fcnet_hiddens": [
                tune.lograndint(64, 512, 2), tune.lograndint(64, 512, 2)],
            "fcnet_activation": "relu",
        },
        "gamma": tune.choice([0.9, 0.95, 0.97, 0.98, 0.99, 0.999, 1]),
        "lr": tune.loguniform(1e-5, 1e-2),
        "grad_clip": tune.loguniform(0.01, 10),
        "train_batch_size": tune.lograndint(8, 256, 2),
        "training_intensity": tune.lograndint(1, 10000),
        "buffer_size": tune.choice([int(5e4), int(2e5), int(5e5), int(1e6)]),
        "exploration_config": {
            "final_epsilon": tune.loguniform(0.02, 0.2),
            "epsilon_timesteps": tune.lograndint(int(1e4), int(1e6)),
        },
    }

    best_hp = [{
        "seed": 131,
        "model": {
            "fcnet_hiddens": [100, 123],
        },
        "gamma": 0.99,
        "lr": 1e-3,
        "grad_clip": 1,
        "train_batch_size": 16,
        "training_intensity": 235,
        "buffer_size": int(1e6),
        "exploration_config": {
            "final_epsilon": 0.1,
            "epsilon_timesteps": int(1e5),
        },
    }]
    
    HyperOpt_algo = HyperOptSearch(
        n_initial_points=1,
        random_state_seed=20210302,
        points_to_evaluate=best_hp
    )
    results = tune.run("DQN", config=config, search_alg=HyperOpt_algo, metric="episode_reward_mean", mode="max")

    ray.shutdown()

If the code snippet cannot be run by itself, the issue will be closed with "needs-repro-script".

• I have verified my script runs in a clean environment and reproduces the issue.
• I have verified the issue also occurs with the latest wheels.

@krfricke Is there any progress on this issue?

I've tried to solve this but I couldn't find its cause.

[krfricke]

I remember looking into and reproducing it, but I didn't find the cause.

It seems though that this is actually a hyperopt problem, as we're just calling the hyperopt API:

# ...

self._hpopt_trials = generate_trials_to_calculate(
                self._points_to_evaluate)

A good next step would be to try to reproduce this with hyper opt only (no Ray Tune). If the error persists, we should open an issue in the hyperopt repo.

[richardliaw]

I've bumped the priority on this issue

Thanks for your help @krfricke @richardliaw !

[krfricke]

Sorry this took so long, but it will be fixed by #18113