bug: python: "Data is not base64 encoded" on server sent events

[yorickvP]

Describe the bug

When starting a response with data: (which happens for Server Sent Events), langfuse tries to parse it as base64 and fails. The test should probably be data:[^\s] so SSE doesn't trigger it.

To reproduce

@observe()
async def test_sse():
    yield "data: {}"
    yield "data: [DONE]"

SDK and container versions

No response

Additional information

No response

Are you interested to contribute a fix for this bug?

No

[dosubot]

The issue you're encountering seems to be related to Langfuse's handling of data prefixed with data: , which is common in Server Sent Events (SSE). The current implementation appears to mistakenly attempt to parse this as base64 encoded data, leading to errors.

A potential solution is to adjust the regex or logic used to identify base64 encoded data so that it doesn't trigger on SSE data. The suggestion to use a regex pattern like data:[^\s] could help differentiate between SSE data and actual base64 encoded data, preventing the erroneous parsing attempt.

If you're looking to implement this fix, you might want to check the specific part of the code where base64 parsing is triggered and adjust the conditions accordingly. Unfortunately, I couldn't find the exact location in the codebase where this logic is implemented, but it might be worth exploring areas where data parsing is handled, especially those related to base64 detection.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[100gle]

I got the same issue. My logger located where the exception raised:

The sever-send event will be treated as base64 data by Langfuse.

[GaryFail]

In my case I was happy enough not to capture the output from my method as all outputs from nodes within my LangGraph graph would supply me all the tracing information I need, or at least enough to see what is going on.

@observe(name="test", capture_output=False)

This of course won't be useful for everyone.

[github-actions]

This issue is stale because it has been open for 30 days with no activity.

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale. Please reopen if the issue persists.

[daydaychen]

I had the same problem.

[rushant001]

In langfuse/_task_manager/media_manager.py, the current logic for detecting media data is too permissive:
if isinstance(data, str) and data.startswith("data:"):
This condition only checks if a string starts with "data:", which can lead to false positives — i.e., non-media strings that happen to begin with "data:" are incorrectly treated as media.

If you are certain that no media data will be sent, you can set the environment variable LANGFUSE_MEDIA_UPLOAD_ENABLED=false to disable media upload functionality and prevent such issues.