Skip to content

Remove hard-coded improvement threshold #344

@cedricdeboom

Description

@cedricdeboom

Assessing whether a program performs better than an other program, is currently governed by a hard-coded threshold of 0.02 in controller.py:

# If the combined_score of this program is significantly better, use it instead
if (
best_by_combined.metrics["combined_score"]
> best_program.metrics["combined_score"] + 0.02
):

What "significantly" means, depends a lot on the nature of the combined_score being used. Therefore, hard-coding 0.02 does not always make sense.

My proposal is to make it a config parameter that can also be put to zero if needed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions