Feedback/Question for “Example: Langfuse Prompt Management with Langchain (Python)”

[Boskicl]

For the prompt ingestion into an App, why is it not being traced under "Prompts" and "linked generations?"

I call:
langfuse_prompt = langfuse.get_prompt("event-planner")

And add it here:

orchestrator = create_agent(
    model=llm,
    system_prompt= langfuse_prompt.prompt,
    tools=AVAILABLE_TOOLS,
    checkpointer=checkpointer,
)

I would expect once you make the call with langfuse.get_prompt that this would signify that this is the prompt for tracing?

Amazing working to the Langfuse team! Love the tracing and everything you guys have done,
-Ljube

[dosubot]

Hi @Boskicl! I'm Dosu and I’m helping the langfuse-docs team.

Retrieving a prompt with langfuse.get_prompt only fetches the prompt data—it doesn't automatically trace its usage or link it to generations in Langfuse. For prompts and generations to be traced and linked, you need to explicitly integrate the prompt into your Langchain workflow with metadata that Langfuse can recognize.

The recommended approach is to convert the Langfuse prompt into a Langchain ChatPromptTemplate and attach the Langfuse prompt object as metadata. This way, when the LLM call is traced, Langfuse can associate the generation with the correct prompt. Here’s an example (in TypeScript, but the concept applies to Python as well):

const langchainChatPrompt = ChatPromptTemplate.fromMessages(
  langfuseChatPrompt.getLangchainPrompt().map((m) => [m.role, m.content])
).withConfig({
  metadata: { langfusePrompt: langfuseChatPrompt },
});

This explicit linking is necessary for prompts to show up under "Prompts" and "linked generations" in the Langfuse UI. For more details and Python-specific examples, check out the Langfuse docs on linking prompts and traces
(source, source).

If you see prompts in tracing but not in prompt management, double-check that you’re attaching the prompt metadata as shown above.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

   

[Boskicl]

Hey @dosu

I keep getting an error with my app since the system_prompt input of langchain_chat_prompt is not proper.

I just need the "text" of the prompt. It works if I do langfuse_chat_prompt.prompt but this doesn't trace the prompt.

Exact python code

# Initialize Langfuse callback handler
langfuse_handler = CallbackHandler()
langfuse_client = Langfuse()
langfuse_chat_prompt = langfuse_client.get_prompt("chat_temp/basic", label="production")

langchain_chat_prompt = ChatPromptTemplate.from_messages(
    langfuse_chat_prompt.get_langchain_prompt()
)
 
langchain_chat_prompt.metadata = {"langfuse_prompt": langfuse_chat_prompt}

orchestrator = create_agent(
    model=llm,
    system_prompt= langchain_chat_prompt,
    tools=AVAILABLE_TOOLS,
    checkpointer=checkpointer,
)

[dosubot]

The error happens because your agent's system_prompt expects a string, but you're passing a ChatPromptTemplate object. Using langfuse_chat_prompt.prompt (the text) works, but doesn't enable prompt tracing in Langfuse.

To trace prompt usage in Langfuse, you need explicit instrumentation—retrieving the prompt alone isn't enough. The recommended approach is to use Langfuse's tracing callbacks when making LLM calls, or manually log the prompt usage to Langfuse when you use the prompt text. There isn't a direct Python example for using the prompt text as system_prompt while also ensuring prompt tracing, and the TypeScript examples don't translate directly if your agent only accepts strings for system_prompt (source, source).

If you want more targeted help, could you share the exact error message and which agent implementation you're using? That way, I can suggest a more specific workaround or integration.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

   

[Boskicl]

Hi @dosu

I am using Ollama as the main LLM (llama3.2:3b) but using ChatOpenAI from langchain_openai and all run through FastAPI.

llm = ChatOpenAI(
    base_url=f"{OLLAMA_BASE_URL}/v1",
    model=OLLAMA_MODEL,
    api_key='ollama',
    temperature=0
)

And this all comes together like before:

orchestrator = create_agent(
    model=llm,
    system_prompt= langchain_chat_prompt,
    tools=AVAILABLE_TOOLS,
    checkpointer=checkpointer,
)

The error is definitely the input for system_prompt

Error:

 Input should be a valid string [type=string_type, input_value=ChatPromptTemplate(input_... additional_kwargs={})]), input_type=ChatPromptTemplate]
new_backend-1  |     For further information visit https://errors.pydantic.dev/2.12/v/string_type
new_backend-1  | content.list[union[str,dict[any,any]]].0.str
new_backend-1  |   Input should be a valid string [type=string_type, input_value=('name', None), input_type=tuple]
new_backend-1  |     For further information visit https://errors.pydantic.dev/2.12/v/string_type
new_backend-1  | content.list[union[str,dict[any,any]]].0.dict[any,any]
new_backend-1  |   Input should be a valid dictionary [type=dict_type, input_value=('name', None), input_type=tuple]
new_backend-1  |     For further information visit https://errors.pydantic.dev/2.12/v/dict_type
new_backend-1  | content.list[union[str,dict[any,any]]].1.str
new_backend-1  |   Input should be a valid string [type=string_type, input_value=('input_variables', []), input_type=tuple]
new_backend-1  |     For further information visit https://errors.pydantic.dev/2.12/v/string_type
new_backend-1  | content.list[union[str,dict[any,any]]].1.dict[any,any]
new_backend-1  |   Input should be a valid dictionary [type=dict_type, input_value=('input_variables', []), input_type=tuple]
new_backend-1  |     For further information visit https://errors.pydantic.dev/2.12/v/dict_type
new_backend-1  | content.list[union[str,dict[any,any]]].2.str
new_backend-1  |   Input should be a valid string [type=string_type, input_value=('optional_variables', []), input_type=tuple]
new_backend-1  |     For further information visit https://errors.pydantic.dev/2.12/v/string_type
new_backend-1  | content.list[union[str,dict[any,any]]].2.dict[any,any]
new_backend-1  |   Input should be a valid dictionary [type=dict_type, input_value=('optional_variables', []), input_type=tuple]
new_backend-1  |     For further information visit https://errors.pydantic.dev/2.12/v/dict_type

[dosubot]

You're seeing that error because system_prompt expects a string, but you're passing a ChatPromptTemplate object. Using the prompt text (e.g., langfuse_chat_prompt.prompt) works, but by itself, it doesn't enable prompt tracing in Langfuse—retrieving or using the prompt text alone isn't enough for Langfuse to link prompts and generations automatically [source].

Currently, there isn't an out-of-the-box Python solution for tracing prompts in Langfuse when only the prompt text can be used as system_prompt. To work around this, you can manually log the prompt usage to Langfuse at the point where you use the prompt text. This means, after you use the prompt in your LLM call, you should create a trace or span in Langfuse and attach the prompt details yourself.

Here's a rough example of how you might do this:

from langfuse import Langfuse

langfuse_client = Langfuse()

# When you use the prompt text:
prompt_text = langfuse_chat_prompt.prompt

# ... use prompt_text in your agent/LLM call ...

# Then, manually log the prompt usage to Langfuse
trace = langfuse_client.create_trace(
    name="llm-call",
    input={"prompt": prompt_text, ...},  # include any relevant input
    metadata={"langfuse_prompt_id": langfuse_chat_prompt.id},
    # ... other fields as needed
)

# Optionally, log the LLM output as well
langfuse_client.create_span(
    trace_id=trace.id,
    name="llm-generation",
    output={"response": llm_response},
    metadata={"prompt_id": langfuse_chat_prompt.id}
)

This manual step links the prompt usage to your traces in Langfuse, even though you're passing only the prompt text to your agent. It's not as seamless as the TypeScript/metadata approach, but it achieves the same result until a more integrated Python solution is available [source].

Keep an eye on the Langfuse docs for updates, as this is a known gap in the current Python integration.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

   

[Boskicl]

@dosu Just to confirm this is a known issue and not something that is supported as an input to system_prompts? Like to make a single call to langfuse which would link it all? And if so, is this on any future work for langfuse team?

If that is the case, that is all I need! Thank you!

[dosubot]

That's correct—this is a known limitation. In Python, you can't link prompt usage to generations in Langfuse via a single call by passing the prompt object to system_prompt; it only accepts a string, and using just the text doesn't automatically trace or link the prompt in Langfuse. There isn't an out-of-the-box solution for this, and recent docs and issues don't show any ongoing or planned work to support single-call prompt tracing/linking in Python. The current approach is to use explicit metadata linking or manual instrumentation as described in the docs (reference, reference). If you need help with workarounds or manual logging, let me know!

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}