Skip to content

Adjust default learning rate for MADLAD #418

Open
@isaac091

Description

@isaac091

There was previously a default learning rate for MADLAD, but all the experiments run to determine the value used LoRA models. Since the learning rate experiments for NLLB and LoRA yielded the same value, it makes sense to reevaluate the default value for MADLAD once we are able to test MADLAD without using LoRA.

Metadata

Metadata

Assignees

No one assigned

    Labels

    optimizationModel training/inferencing optimizationpipeline 4: trainIssue related to training a model.

    Type

    No type

    Projects

    Status

    📋 Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions