Add an option to link an example to prompt from trace

[orgroman]

Feature request

Since langsmith is already detecting correctly that I'm using a prompt template from the private hub please make it so that I can easily add an example to the prompt playground from the trace view just like I can do when running the prompt directly.

Motivation

This will make things a lot more easier to us since prompt engineers could easily add an example to the prompt playground from a given trace and he or she can use it to optimize the prompt

[hinthornw]

Could you elaborate a bit more on what your desired UX is?

You can add any run to a dataset, and you can open datasets in the prompt playground when iterating on a prompt.

[orgroman]

Thank you for your response! I appreciate the ability to add a run to a dataset, but I've noticed that it doesn't retain the input template of the original prompt. For example, if I've defined a template with variables like format_instructions and others, it merges these variables into one input when adding an example to the dataset, which doesn't align with my intended workflow.

Additionally, since we're discussing the prompt playground, it would be helpful to have the option to switch the prompt template type between chat and completion modes and vice versa. It would also be great to be able to add an output schema after the prompt is created, through the edit functionality, rather than only during the initial creation.

[hinthornw]

Hi @orgroman - are you using prompt runs? Or looking just at the LLM runs?

It sounds like what you want is the prompt run, which keeps the template variables as described while also supporting the playground