Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create function for building best network #10

Open
craffel opened this issue Apr 18, 2016 · 0 comments
Open

Create function for building best network #10

craffel opened this issue Apr 18, 2016 · 0 comments

Comments

@craffel
Copy link
Owner

craffel commented Apr 18, 2016

Would look something like this:

def construct_best_network(model_file, trial_glob):
    """
    Constructs the best-performing model according to the supplied
    hyperparameter optimization trials, and populates the parameters with the
    provided model file.

    Parameters
    ----------
    model_file : str
        Path to h5 file holding the parameters for this model.
    trial_glob : str
        Glob of h5 trial files.

    Returns
    -------
    layers : list of lasagne.layers.Layer
        Layers comprising the model.
    """
    # Load in all parameter optimization trials
    trial_files = glob.glob(trial_glob)
    trials = [deepdish.io.load(f) for f in trial_files]
    # Ignore trials with NaN objective
    trials = [t for t in trials if not np.isnan(t['best_objective'])]
    # Get the hyperparameters for the trial with the lowest objective value
    best_trial = sorted(trials, key=lambda t: t['best_objective'])[0]
    hyperparameters = best_trial['hyperparameters']
    # Load in the pre-trained parameters for the best performing models
    network_params = deepdish.io.load(model_file)
    # Create dictionary mapping network structure names to build functions
    build_network = {
        'pse_big_filter': build_pse_net_big_filter,
        'pse_small_filters': build_pse_net_small_filters,
        'dhs_big_filter': build_pse_net_big_filter,
        'dhs_small_filters': build_pse_net_small_filters}
    # PSE and DHS nets have different hyperparameters, so we must construct
    # kwargs based on what appears in hyperparameters
    kwargs = {
        # PSE trials do not have this hyperparameter by default, so as in
        # experiment_utils.run_trial, we must set the default value
        'downsample_frequency': hyperparameters.get('downsample_frequency', 1)}
    if 'n_attention' in hyperparameters:
        kwargs['n_attention'] = hyperparameters['n_attention']
    if 'n_conv' in hyperparameters:
        kwargs['n_conv'] = hyperparameters['n_conv']
    layers = build_network(
        (None, 1, None, feature_extraction.N_NOTES),
        # We will supply placeholders here but load in the values below
        np.zeros((1, feature_extraction.N_NOTES), theano.config.floatX),
        np.ones((1, feature_extraction.N_NOTES), theano.config.floatX),
        **kwargs)
    # Load in network parameter values
    lasagne.layers.set_all_param_values(
        layers[-1], network_params['X'])
    return layers

Would be used in precompute.py for a few of the experiments, and in match.py.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant