plots: improve vs code/studio integration

[dberenbaum]

This is an issue to track potential plots changes to streamline some of the plots handling happening in VS Code and Studio. It should enable VS Code and Studio to drop custom rendering for things like setting strokeDash for filename and field, and it should enable them to take advantage of DVC plots improvements like those in #8900.

The rest of this issue is copied from treeverse/vscode-dvc#3130 (comment)_

---

There's a rough POC of the ideas above in https://github.com/iterative/dvc/tree/plots-fields and https://github.com/iterative/dvc-render/tree/plots-fields.

These changes updates output to include these new fields:

```json
"dvc_id": "workspace::train/acc.tsv::train/acc",
"dvc_rev": "workspace",
"dvc_filename": "train/acc.tsv",
"dvc_field": "train/acc",
"dvc_source": "train/acc.tsv::train/acc"
```

• Incorporates these into templates, using dvc_rev for color and dvc_source for strokeDash.
• Drops data_version_info.

Here's an example of dvc plots diff --split --json output:

{
  "dvc.yaml::Accuracy": [
    {
      "type": "vega",
      "revisions": [
        "HEAD",
        "workspace"
      ],
      "content": {
        "$schema": "https://vega.github.io/schema/vega-lite/v5.json",
        "data": {
          "values": "<DVC_METRIC_DATA>"
        },
        "title": "dvc.yaml::Accuracy",
        "width": 300,
        "height": 300,
        "params": [
          {
            "name": "smooth",
            "value": 0.001,
            "bind": {
              "input": "range",
              "min": 0.001,
              "max": 1,
              "step": 0.001
            }
          }
        ],
        "layer": [
          {
            "mark": "line",
            "encoding": {
              "x": {
                "field": "step",
                "type": "quantitative",
                "title": "step"
              },
              "y": {
                "field": "dvc_inferred_y_value",
                "type": "quantitative",
                "title": "accuracy",
                "scale": {
                  "zero": false
                }
              },
              "color": {
                "field": "dvc_rev",
                "type": "nominal"
              },
              "strokeDash": {
                "field": "dvc_source",
                "type": "nominal"
              }
            },
            "transform": [
              {
                "loess": "dvc_inferred_y_value",
                "on": "step",
                "groupby": [
                  "dvc_rev",
                  "dvc_source"
                ],
                "bandwidth": {
                  "signal": "smooth"
                }
              }
            ]
          },
          {
            "mark": {
              "type": "point",
              "tooltip": {
                "content": "data"
              }
            },
            "encoding": {
              "x": {
                "field": "step",
                "type": "quantitative",
                "title": "step"
              },
              "y": {
                "field": "dvc_inferred_y_value",
                "type": "quantitative",
                "title": "accuracy",
                "scale": {
                  "zero": false
                }
              },
              "color": {
                "field": "dvc_rev",
                "type": "nominal"
              },
              "strokeDash": {
                "field": "dvc_source",
                "type": "nominal"
              }
            }
          }
        ]
      },
      "datapoints": {
        "workspace": [
          {
            "step": "0",
            "train/acc": "0.5262",
            "dvc_inferred_y_value": "0.5262",
            "rev": "workspace::train/acc.tsv::train/acc",
            "dvc_id": "workspace::train/acc.tsv::train/acc",
            "dvc_rev": "workspace",
            "dvc_filename": "train/acc.tsv",
            "dvc_field": "train/acc",
            "dvc_source": "train/acc.tsv::train/acc"
          },
...

[mattseddon]

Will the VS Code extension not still need to specify the strokeDash/color for files/experiments respectively? We have to display that information in various places in the UI:

Screen.Recording.2023-02-14.at.3.47.52.pm.mov

Screen.Recording.2023-02-14.at.3.53.13.pm.mov

If that information is auto-generated won't we have to introduce the legends within the plots and drop all of the info from the rest of the UI? Is that the plan?

Note: We also apply a shape transformation under the circumstances that there are multiple fields accessed within a file.

[dberenbaum]

Will the VS Code extension not still need to specify the strokeDash/color for files/experiments respectively?

• strokeDash/ files: My hope is VS Code does not need to touch it. Use dvc_... fields from https://github.com/iterative/dvc/pull/9019 dvc-render#119 should be able to replicate what you already do.
• color / experiments: VS Code will still need to override colors I think to make them stable for each experiment. However, I think at least the revision/experiment will be available as dvc_rev, so you wouldn't need to manipulate the underlying data, just update the template.

Note: We also apply a shape transformation under the circumstances that there are multiple fields accessed within a file.

• I think it should not be hard to update Use dvc_... fields from https://github.com/iterative/dvc/pull/9019 dvc-render#119 to do this.

Does it sound right to you @mattseddon? Does it seem worth it?

[mattseddon]

Ok, so IIUC. The main change here is that we will no longer have to manipulate the datapoints that we are passing to Vega but we will still have to manipulate the template with strokeDash/color/shape/legend information. At the moment I think we will still have to cycle through every datapoint to gather the aggregate data required to decide on the strokeDash/shape. We can potentially push this aggregation back into DVC if it is provided on a per-revision basis (we cache the information on the extension side rather than always requesting every revision that we are trying to display).

I think it should not be hard to update treeverse/dvc-render#119 to do this.
Does it sound right to you @mattseddon? Does it seem worth it?

The more centralised we can make things the better.

[dberenbaum]

@mattseddon @yathomasi #9019 and treeverse/dvc-render#119 should be ready for review. You need to try:

pip install git+https://github.com/iterative/dvc.git@plots-fields
pip install git+https://github.com/iterative/dvc-render.git@plots-fields

There's a summary of the new data format at the top of this issues or in #9019 (comment), and there's a summary of template updates in treeverse/dvc-render#119 (comment).

[yathomasi]

Hey @dberenbaum, I tried this, and here are some visuals.

Before	After

I have a few confusion/queries:

• It looks like we don't have use shape on the Studio, how do we use that
• It has affected the Confusion Matrix
• Also, with different fields( dvc_id, dvc_rev, rev), I am a little confused about what should be used where.

[dberenbaum]

Thanks @yathomasi and sorry for taking so long getting back to this.

• It looks like we don't have use shape on the Studio, how do we use that

When the y values have multiple field/column names, VS Code uses shapes to distinguish these, and this PR does the same. I don't think you should need to do anything special for them.

For example, take a look at the following plot definition and plot:

plots:
  - acc_v_loss:
      x: step
      y:
        training/plots/metrics/train/acc.tsv: acc
        training/plots/metrics/test/acc.tsv: acc
        training/plots/metrics/train/loss.tsv: loss
        training/plots/metrics/test/loss.tsv: loss
      y_label: Accuracy or loss

See https://github.com/dberenbaum/vscode-dvc-demo/tree/acc_v_loss if you want to try it out.

---

• It has affected the Confusion Matrix

🤔 I can't reproduce it locally:

The facet spec in the updated template looks like:

        "facet": {
            "field": "dvc_id",
            "type": "nominal",
            "header": {"title": False},
        },

Do you know if Studio manipulates that? I don't think it should have to.

---

• Also, with different fields( dvc_id, dvc_rev, rev), I am a little confused about what should be used where.

• dvc_id: This is the same as rev (see below), and it is a concatenation of revision::filename::field (if any of those don't exist for the plot, they won't be included).
• dvc_rev: This is the actual revision, not concatenated with any other info.
• rev: This is a legacy field (same as dvc_id). I renamed to dvc_id to be more accurate and consistent but can't drop this field without breaking older templates that reference it.