dvc exp push: pushing/pulling cache fails if params.yaml not in workspace

[dberenbaum]

Bug Report

Description

If there is no params.yaml in the workspace and interpolation is used in dvc.yaml, dvc exp push will fail to push the cache for the experiments.

Reproduce

Fork and clone https://github.com/dberenbaum/params-test and then run:

dvc exp run
rm params.yaml
dvc exp push origin

You will see an error like ERROR: failed to push cache: failed to parse 'stages.params_test.cmd' in 'dvc.yaml': Could not find 'params'.

Expected

Since params.yaml is present in the experiment, it should be possible to push it. Can we interpolate using the state of the experiment rather than the workspace?

[dberenbaum]

It seems very loosely related to #9754 in that we aren't respecting what's in the revision but instead are using the workspace.

[daavoo]

For the record in case someone encounters this in the context of using Hydra integration:

I missed the Hydra part before. That might be easier to handle as a special case in DVC.
And even though it still adds some friction, assuming the Hydra stuff is present in the workspace, you could do dvc exp run --dry as a fallback.
That would create the params.yaml in the workspace and allow to run dvc commands that would previously fail at the interpolation stage

[dberenbaum]

This can also happen during pull, and not only on the first run, since parameters may change frequently (esp with hydra), and any missing parameter in the workspace causes a failure. See https://discord.com/channels/485586884165107732/485596304961962003/1137288948914339860.

[daavoo]

This can also happen during pull, and not only on the first run, since parameters may change frequently (esp with hydra), and any missing parameter in the workspace causes a failure. See https://discord.com/channels/485586884165107732/485596304961962003/1137288948914339860.

Should we compose_and_dump by default on dvc exp commands?

[dberenbaum]

This can also happen during pull, and not only on the first run, since parameters may change frequently (esp with hydra), and any missing parameter in the workspace causes a failure. See https://discord.com/channels/485586884165107732/485596304961962003/1137288948914339860.

Should we compose_and_dump by default on dvc exp commands?

Do you mean in the workspace? I don't think it would solve the issue mentioned in discord since it seems like whether the params exist depends on the config in the specific experiment, and it may also mess up the user's workspace.

[Danila89]

Are there any updates on this issue? Garbage collection is also affected: dvc gc -a -c --remote remote-name --all-experiments --force fails with similar error

[dberenbaum]

@Danila89 I don't have any updates unfortunately. Have you tried adding params.yaml to your workspace? I think it should work once you have a params.yaml file with the expected parameters.