Comments on objects across the Langfuse UI

[marcklingen]

Summary

Add comments feature on traces/observations/sessions/prompts within the Langfuse UI.

Background

Many teams run internal workflows based on different objects in the Langfuse UI. For example, they investigate reasons for bad user feedback or low evaluation results.

Currently, these workflows can be organized using tags. However, findings need to be stored outside of Langfuse (Linear/Jira) or as a score comment within Langfuse (which is not the intended use).

We can easily build this if there is enough demand (please upvote if you are interested). If you have any specific requirements or wishes, please feel free to share them.

[Manouchehri]

It'd be sick if comment notifications could be hooked up to Slack eventually (like GitHub's Slack app).

[marcklingen]

This was added by @marliessophie recently, feel free to share any feedback!

https://langfuse.com/changelog/2024-08-20-comments

[Zugunruhekami]

Thanks @marcklingen @marliessophie for the great initiative and work! May I ask if there is plan to allow easier search of the commented objects (traces, sessions, generations, prompts)? Currently when I leave comments, it doesn't notify the members in the team to view the comments. Hence a filter or a new column in trace view will be great to show all the annotated traces in one place for us to check.

Another way is to automatically attach a tag to a trace when there is inserted annotation from Langfuse UI.

We also use Jira a lot hence current practice is to leave trace links in our jira tickets. Will be helpful to have a way to bind the two systems in an easy way.

[marcklingen]

Thanks for sharing! Can you add these as independent feature requests? Both notifications and filtering are really interesting extensions of comments