Fix YAML referencing to allow better CLI overrides

[abhi-mosaic]

Currently we rely on YAML anchors to copy things like max_seq_len into many places.

But since we use OmegaConf to read our YAMLs, we can instead use the ${max_seq_len} syntax.

This allows CLI to override the anchored values like data_remote, data_local, max_seq_len, etc. that could not be done before. Previously, you would have to override every materialized value like train_loader.dataset.remote.

There are a lot of files here, but I'm going to check the high priority ones 1 by 1 to make sure nothing broke. And then assume that YAMLs within a dir (like yamls/mosaic_gpt/... all work).

[dakinggg]

yesss thank you. LGTM, assuming you checked a few and the syntax works.

[dblalock]

So I'm kind of confused why we still use YAML anchors for the run_name in the BERT configs.

• ./examples/bert/yamls/finetuning/glue/mcloud_run.yaml: base_run_name: *run_name
• ./examples/bert/yamls/finetuning/mcloud_run.yaml: run_name: *run_name
• ./examples/bert/yamls/main/mcloud_run_a100_40gb.yaml: run_name: *run_name
• ./examples/bert/yamls/main/mcloud_run_a100_80gb.yaml: run_name: *run_name

But seems like there are no issues besides those we already flagged.

In more detail, I wrote a script to parse the old and new versions via omegaconf, resolve() them, and then check that they're the same. It correctly found the new null that Daniel flagged and said everything else is the same*
so I kinda believe it.

*Except that the BERT yamls can't resolve anything because you need to write out the parameters: as a separate yaml file for it to work. (Yet another reason we need to indirect to the other config files).

[abhi-mosaic]

So I'm kind of confused why we still use YAML anchors for the run_name in the BERT configs.

I agree, we shouldn't have these. But I think I need to fix the BERT training scripts to read in COMPOSER_RUN_NAME correctly to fix. I will look into this, but in a separate PR.