Skip to content

Refactor the concept of public and private test set #1474

Closed
@mfeurer

Description

@mfeurer

Auto-sklearn was built for the AutoML challenges where there were public and private test sets by the names "validation set" and "test set". Since these challenges we have substantially refactored Auto-sklearn and no longer allow the user to pass in the public test set aka "validation set". This leads to confusion as "validation set" is really a misnomer, as inside a machine learning system it usually denotes the dataset on which a model is selected. Therefore we should drop all references that still exist to this old concept of a "validation set", as already being done by @eddiebergman in #1434, which should only be in the evaluation submodule`.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions