Replaces plots dvc_data_version_info with flat fields to be used by VS Code and Studio

[dberenbaum]

Fixes #9018.

Without this PR, plots show/diff --json adds a nested dvc_data_version_info field that VS Code uses to do custom rendering. This PR drops dvc_data_version_info and adds these new flat fields to the plots output data:

"dvc_id": "workspace::train/acc.tsv::train/acc",
"dvc_rev": "workspace",
"dvc_filename": "train/acc.tsv",
"dvc_field": "train/acc",
"dvc_source": "train/acc.tsv::train/acc"

Why?

• Flat fields are easier to use in vega-lite templates.
• Both VS Code and Studio can reuse these fields for their own custom rendering.
• Without this PR, dvc_data_version_info is only added if --json is used. With this PR, there is parity between all plots output formats.

Example output of dvc plots show --split --json:

Before this PR

      "datapoints": {
        "workspace": [
      {
        "step": "0",
        "train/acc": "0.2707333333333333",
        "dvc_inferred_y_value": "0.2707333333333333",
        "dvc_data_version_info": {
          "filename": "training/plots/metrics/train/acc.tsv",
          "field": "train/acc"
        },
    },

With this PR

      "datapoints": {
        "workspace": [
          {
            "step": "0",
            "train/acc": "0.5262",
            "dvc_inferred_y_value": "0.5262",
            "rev": "workspace::train/acc.tsv::train/acc",
            "dvc_id": "workspace::train/acc.tsv::train/acc",
            "dvc_rev": "workspace",
            "dvc_filename": "train/acc.tsv",
            "dvc_field": "train/acc",
            "dvc_source": "train/acc.tsv::train/acc"
          },

[skshetry]

@dberenbaum, what's the difference between rev and dvc_id?

[shcheklein]

@dberenbaum before merging this, please ping me - I can do some followup changes on the VS Code if needed.

[dberenbaum]

@dberenbaum before merging this, please ping me - I can do some followup changes on the VS Code if needed.

👍 No plans to merge imminently (it's still in rough shape). I wanted to open the PR for discussion and see if it would be useful for VS Code and Studio.

[dberenbaum]

@dberenbaum, what's the difference between rev and dvc_id?

Nothing. rev is a remnant that is used by old templates, so I didn't want to drop it.

[daavoo]

If we include those fields in the datapoints we probably need to reconsider the current tooltip

https://github.com/iterative/dvc-render/blob/2aa47c86a4a364b0d410e90671b6e57de045b9d7/src/dvc_render/vega_templates.py#L526

[dberenbaum]

If we include those fields in the datapoints we probably need to reconsider the current tooltip

See also https://github.com/iterative/studio/pull/5015/files#r1100509945.

[codecov]

Codecov Report

Patch coverage: 100.00% and project coverage change: -0.02 ⚠️

Comparison is base (17f523b) 92.93% compared to head (c93df68) 92.92%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #9019      +/-   ##
==========================================
- Coverage   92.93%   92.92%   -0.02%     
==========================================
  Files         456      456              
  Lines       36894    36866      -28     
  Branches     5323     5314       -9     
==========================================
- Hits        34287    34257      -30     
- Misses       2085     2088       +3     
+ Partials      522      521       -1     

Impacted Files	Coverage Δ
dvc/commands/plots.py	87.24% <ø> (-1.71%)	⬇️
dvc/render/__init__.py	100.00% <100.00%> (ø)
dvc/render/convert.py	87.09% <100.00%> (ø)
dvc/render/converter/vega.py	91.83% <100.00%> (+0.17%)	⬆️
tests/integration/plots/test_plots.py	94.08% <100.00%> (ø)
tests/unit/render/test_convert.py	100.00% <100.00%> (ø)
tests/unit/render/test_match.py	100.00% <100.00%> (ø)
tests/unit/render/test_vega_converter.py	100.00% <100.00%> (ø)

... and 1 file with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

[mattseddon]

If we include those fields in the datapoints we probably need to reconsider the current tooltip

See also https://github.com/iterative/studio/pull/5015/files#r1100509945.

And iterative/vscode-dvc#3119

[dberenbaum]

Closing since it doesn't seem we have time or enough benefit for this.