crewAI+fastAPI

[cloud-java]

Feature Area

Core functionality

Is your feature request related to a an existing bug? Please link it here.

Subject: How to Integrate CrewAI with FastAPI for Streaming Output

Description:

Hello CrewAI team!

First, thank you for this amazing framework! I'm currently working on integrating CrewAI with a FastAPI backend and I need to implement streaming output for the AI responses.

My Use Case:

• I want to create a FastAPI endpoint that uses CrewAI agents/crews
• Instead of waiting for the complete response, I need to stream the output progressively to the frontend
• This is crucial for better user experience with longer-running tasks

What I've Tried:
I've looked through the documentation and examples but couldn't find specific guidance on:

• How to properly handle CrewAI with async FastAPI
• How to access intermediate results or stream output chunks
• Best practices for integration with web frameworks

Questions:

1. What would be the recommended approach to integrate CrewAI with FastAPI for real-time streaming?
2. Are there any examples of web framework integrations planned?

I would greatly appreciate any guidance, code examples, or pointers to relevant documentation. Thank you for your time and contribution to the AI community!

Describe the solution you'd like

Subject: How to Integrate CrewAI with FastAPI for Streaming Output

Description:

Hello CrewAI team!

First, thank you for this amazing framework! I'm currently working on integrating CrewAI with a FastAPI backend and I need to implement streaming output for the AI responses.

My Use Case:

• I want to create a FastAPI endpoint that uses CrewAI agents/crews
• Instead of waiting for the complete response, I need to stream the output progressively to the frontend
• This is crucial for better user experience with longer-running tasks

What I've Tried:
I've looked through the documentation and examples but couldn't find specific guidance on:

• How to properly handle CrewAI with async FastAPI
• How to access intermediate results or stream output chunks
• Best practices for integration with web frameworks

Questions:

1. What would be the recommended approach to integrate CrewAI with FastAPI for real-time streaming?
2. Are there any examples of web framework integrations planned?

I would greatly appreciate any guidance, code examples, or pointers to relevant documentation. Thank you for your time and contribution to the AI community!

Describe alternatives you've considered

No response

Additional context

No response

Willingness to Contribute

Yes, I'd be happy to submit a pull request

[cloud-java]

Subject: Request to Merge Streaming Support Branch

Hello again @joaomdmoura @joaoigm ! 👋

I wanted to express my heartfelt thanks for creating the devin/1760962840-fastapi-streaming-support branch to address this feature! What truly amazed me was how incredibly fast you responded and delivered – it far exceeded my expectations!

I've taken the time to review the implementation in the branch, and I'm genuinely impressed by its clean and powerful design. The callback handlers and example code you provided are exceptionally clear and practical, offering a seamless path for integrating CrewAI with FastAPI streaming outputs.

Now that the feature has been fully developed in the branch, would it be possible to consider merging these changes into the main branch? This would make it significantly easier for the wider community to access and leverage this functionality without needing to switch to a feature branch.

Integrating streaming support into the main release will undoubtedly enhance CrewAI's capabilities for building interactive web applications, opening up many new possibilities for the community.

Once again, thank you for your outstanding work and dedication – your expertise and responsiveness to community needs are truly commendable and deeply appreciated!

[Vidit-Ostwal]

Hi @cloud-java, There is something called event listeners which I think you should definitely check that out, with that I think you don't need this particular PR to get merged.
Here is the documentation link: https://docs.crewai.com/en/concepts/event-listener#event-listeners
Check this particular event : LLMStreamChunkEvent

[cloud-java]

Good morning! @Vidit-Ostwal . Checking this issue right after waking up has become my new routine, and I'm thrilled to see the response from the CrewAI contributors!

Thank you for the helpful suggestions. However, as a newcomer to the AI agent field, I must admit I'd need more concrete guidance. Would it be possible to provide a complete working example of how to use the LLMStreamChunkEvent within the FastAPI framework?

I'm truly grateful for your outstanding contributions and deeply touched by the CrewAI community's enthusiasm. A comprehensive example would be invaluable for learners like me. Your warm support truly deserves appreciation!

[Vidit-Ostwal]

No worries,

Can you add this on top

from crewai.events import (
    LLMStreamChunkEvent
)
from crewai.events import BaseEventListener

class MyCustomListener(BaseEventListener):
    def __init__(self):
        super().__init__()

    def setup_listeners(self, crewai_event_bus):
        @crewai_event_bus.on(LLMStreamChunkEvent)
        def on_llm_stream_chunk(source, event):
                  print(event)

my_listener = MyCustomListener()

[cloud-java]

Thank you very much for your help. With your assistance, I have successfully implemented this function. @Vidit-Ostwal

[Vidit-Ostwal]

Thank you very much for your help. With your assistance, I have successfully implemented this function. @Vidit-Ostwal

Awesome, Glad I can help @cloud-java

[ssssshql]

@cloud-java Hello, I am currently using FastAPI and Crewai, and I am encountering an issue: there seems to be no property like message_id in LLMStreamChunkEvent that allows me to determine which streams belong to the same message. Currently, I am using task_id and agent_id, but I have found that in the same task, the same agent may speak twice, which may cause two messages to merge together. I see that you seem to have implemented this feature. Can you share how to solve this problem

[Vidit-Ostwal]

Hi @ssssshql, minding checking out this feature branch

#3842

Whether this works for you or not
TQ

Just do a pip install git+https://github.com/Vidit-Ostwal/crewAI.git@vo/feat/FixingStreamID
LMK this works or not