-
Notifications
You must be signed in to change notification settings - Fork 5.9k
Description
Description
When a task has a guardrail configured, TaskOutput.pydantic is None on the first guardrail invocation, but correctly parsed on retry attempts. This inconsistency makes it difficult to write guardrail functions that need to access the structured Pydantic output.
The root cause is in task.py:
- First attempt (lines 677-680): When guardrails exist,
_export_output()is intentionally skipped, settingpydantic_output = None - Retry attempts (line 1151):
_export_output()is always called, properly parsing the Pydantic model
This creates inconsistent behavior where the same guardrail function receives differentTaskOutputstructures depending on whether it's the first or subsequent attempt.
Steps to Reproduce
- Create a Task with
output_pydanticset to a Pydantic model - Add a guardrail function that accesses
task_output.pydantic - Run the crew
- Observe that on the first guardrail call,
task_output.pydanticisNone - Force a retry (return
Falsefrom guardrail) - Observe that on subsequent calls,
task_output.pydanticis correctly populated
Expected behavior
TaskOutput.pydantic should be consistently parsed and available on all guardrail invocations, including the first attempt. The guardrail function should receive the same TaskOutput structure regardless of whether it's the first attempt or a retry.
Screenshots/Code snippets
Minimal reproduction:
from crewai import Agent, Crew, Task, TaskOutput
from pydantic import BaseModel
class MyOutput(BaseModel):
message: str
status: str
def my_guardrail(task_output: TaskOutput) -> tuple[bool, TaskOutput]:
print(f"Pydantic value: {task_output.pydantic}") # None on first call!
print(f"Raw value: {task_output.raw}") # Has the JSON string
if task_output.pydantic is None:
# First attempt - pydantic not parsed!
return False, "Pydantic was None"
return True, task_output
agent = Agent(role="Test", goal="Test", backstory="Test")
task = Task(
description="Return a message",
expected_output="JSON with message and status",
output_pydantic=MyOutput,
guardrail=my_guardrail,
agent=agent,
)
Operating System
Ubuntu 24.04
Python Version
3.12
crewAI Version
1.9.3
crewAI Tools Version
1.9.3
Virtual Environment
Venv
Evidence
First attempt (_execute_core, lines 677-680):
if not self._guardrails and not self._guardrail:
pydantic_output, json_output = self._export_output(result)
else:
pydantic_output, json_output = None, None # ← SKIPPED!
task_output = TaskOutput(
...
pydantic=pydantic_output, # ← None on first attempt
...
)
Retry attempts (_invoke_guardrail_function, line 1151):
# After retry, parsing IS done:
pydantic_output, json_output = self._export_output(result) # ← ALWAYS CALLED
task_output = TaskOutput(
...
pydantic=pydantic_output, # ← Properly parsed on retries
...
)
Possible Solution
Remove the conditional skip of _export_output() on the first attempt. The parsing should always occur before the guardrail is invoked:
# Lines 677-680 should be changed from:
if not self._guardrails and not self._guardrail:
pydantic_output, json_output = self._export_output(result)
else:
pydantic_output, json_output = None, None
# To simply:
pydantic_output, json_output = self._export_output(result)
This ensures consistent behavior across all attempts and allows guardrail functions to reliably access the structured Pydantic output.
Additional context
Additional Context:
- This inconsistency forces workarounds in guardrail functions, such as manually parsing
task_output.rawwith JSON on the first attempt - The current behavior seems intentional (perhaps for performance?), but it breaks the contract that
output_pydanticshould provide structured access in guardrails - Affects both single guardrail (
guardrail=) and multiple guardrails (guardrails=[]) configurations - The async version (
_ainvoke_guardrail_function) likely has the same issue