Update Langsmith for tracing

[karthik18495]

GitHub Issue: Migrate LLM Chain Tracing from Langsmith to Langfuse or OpenTelemetry

Issue Title:
Migrate LLM Chain Tracing from Langsmith to Langfuse or OpenTelemetry

---

Description:

Currently, our system is using Langsmith for tracing LLM chains, but since Langsmith has stopped offering free traces, we need to transition to a more sustainable tracing solution. The proposed solution is to replace Langsmith with either Langfuse or OpenTelemetry, both of which offer long-term support and scalability for tracking and monitoring LLM operations. The frameworks can be self-hosted.

A long-term goal would be to display the entire trace for each conversation directly within the chat interface, providing better visibility and traceability for users. However, for the current implementation, the immediate task is to replace Langsmith with telemetry-based solutions to ensure continuity of tracing and monitoring.

---

Tasks:

1. Evaluate Langfuse vs OpenTelemetry:

   • Assess the features, ease of integration, and long-term sustainability of Langfuse and OpenTelemetry.
   • Select the most suitable solution based on system needs, including tracing, monitoring, and scalability.

2. Update Tracing Middleware:

   • Replace the existing Langsmith tracing middleware with the selected solution (Langfuse or OpenTelemetry).
   • Ensure that the new solution captures key LLM chain trace data (e.g., input/output, intermediate steps, error handling, etc.).

3. Modify Configuration:

   • Update configuration settings to support the new telemetry tool.
   • Ensure that the new tool is compatible with the current logging and monitoring infrastructure.

4. Testing and Validation:

   • Test the new telemetry system to ensure that all tracing and monitoring functionalities are working as expected.
   • Validate that the new telemetry tool provides complete traceability and accurate monitoring of LLM chains.

5. Long-Term Goal Planning:

   • Plan the implementation of displaying the entire trace within the chat interface for each conversation, allowing users to directly view and analyze the LLM chain interactions.

---

Proposed Code Changes:

• Replace Langsmith Tracing Middleware:
  Modify the current tracing logic to use either Langfuse or OpenTelemetry as the backend for capturing trace data.

• Telemetry Integration:
  Add OpenTelemetry's SDK or Langfuse integration in the necessary parts of the LLM chain to ensure tracing events are logged.

• Update Configuration Files:
  Modify configuration settings to switch from Langsmith API calls to OpenTelemetry or Langfuse configurations.

---

Priority:

High - Continuity of tracing is critical for ongoing system monitoring and debugging. This needs to be addressed urgently to prevent gaps in the tracing workflow.