Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release 2.7.0 #1540

Merged
merged 70 commits into from
Dec 12, 2024
Merged

Release 2.7.0 #1540

merged 70 commits into from
Dec 12, 2024

Conversation

dirkbrnd
Copy link
Contributor

Description

  • Add multi-modal support for agents. Image and Video as model output can be added to agents.
  • Add toolkits for Replicate, FAL and luma

Type of change

Please check the options that are relevant:

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Model update
  • Infrastructure change

Checklist

  • My code follows Phidata's style guidelines and best practices
  • I have performed a self-review of my code
  • I have added docstrings and comments for complex logic
  • My changes generate no new warnings or errors
  • I have added cookbook examples for my new addition (if needed)
  • I have updated requirements.txt/pyproject.toml (if needed)
  • I have verified my changes in a clean environment

Additional Notes

Include any deployment notes, performance implications, or other relevant information:

@manthanguptaa
Copy link
Contributor

manthanguptaa commented Dec 11, 2024

Ran into this issue when running the DALLE image agent on the playground. This happens on the 2nd run in the same session

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/responses.py", line 259, in __call__
    await wrap(partial(self.listen_for_disconnect, receive))
  File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/responses.py", line 255, in wrap
    await func()
  File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/responses.py", line 232, in listen_for_disconnect
    message = await receive()
  File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/uvicorn/protocols/http/httptools_impl.py", line 555, in receive
    await self.message_event.wait()
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/asyncio/locks.py", line 226, in wait
    await fut
asyncio.exceptions.CancelledError: Cancelled by cancel scope 10d3b8640

During handling of the above exception, another exception occurred:

  + Exception Group Traceback (most recent call last):
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/uvicorn/protocols/http/httptools_impl.py", line 401, in run_asgi
  |     result = await app(  # type: ignore[func-returns-value]
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
  |     return await self.app(scope, receive, send)
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/fastapi/applications.py", line 1054, in __call__
  |     await super().__call__(scope, receive, send)
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/applications.py", line 113, in __call__
  |     await self.middleware_stack(scope, receive, send)
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/middleware/errors.py", line 187, in __call__
  |     raise exc
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/middleware/errors.py", line 165, in __call__
  |     await self.app(scope, receive, _send)
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/middleware/cors.py", line 93, in __call__
  |     await self.simple_response(scope, receive, send, request_headers=headers)
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/middleware/cors.py", line 144, in simple_response
  |     await self.app(scope, receive, send)
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
  |     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
  |     raise exc
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
  |     await app(scope, receive, sender)
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/routing.py", line 715, in __call__
  |     await self.middleware_stack(scope, receive, send)
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/routing.py", line 735, in app
  |     await route.handle(scope, receive, send)
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/routing.py", line 288, in handle
  |     await self.app(scope, receive, send)
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/routing.py", line 76, in app
  |     await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
  |     raise exc
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
  |     await app(scope, receive, sender)
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/routing.py", line 74, in app
  |     await response(scope, receive, send)
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/responses.py", line 259, in __call__
  |     await wrap(partial(self.listen_for_disconnect, receive))
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 763, in __aexit__
  |     raise BaseExceptionGroup(
  | exceptiongroup.ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/responses.py", line 255, in wrap
    |     await func()
    |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/responses.py", line 244, in stream_response
    |     async for chunk in self.body_iterator:
    |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/concurrency.py", line 62, in iterate_in_threadpool
    |     yield await anyio.to_thread.run_sync(_next, as_iterator)
    |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/anyio/to_thread.py", line 56, in run_sync
    |     return await get_async_backend().run_sync_in_worker_thread(
    |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 2441, in run_sync_in_worker_thread
    |     return await future
    |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 943, in run
    |     result = context.run(func, *args)
    |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/concurrency.py", line 51, in _next
    |     return next(iterator)
    |   File "/Users/manthangupta/Desktop/lab/phidata/phi/playground/router.py", line 93, in chat_response_streamer
    |     for run_response_chunk in run_response:
    |   File "/Users/manthangupta/Desktop/lab/phidata/phi/agent/agent.py", line 1900, in _run
    |     self.write_to_storage()
    |   File "/Users/manthangupta/Desktop/lab/phidata/phi/agent/agent.py", line 739, in write_to_storage
    |     self._agent_session = self.storage.upsert(session=self.get_agent_session())
    |   File "/Users/manthangupta/Desktop/lab/phidata/phi/agent/agent.py", line 597, in get_agent_session
    |     agent_data=self.get_agent_data(),
    |   File "/Users/manthangupta/Desktop/lab/phidata/phi/agent/agent.py", line 577, in get_agent_data
    |     agent_data["images"] = [img.model_dump() for img in self.images]
    |   File "/Users/manthangupta/Desktop/lab/phidata/phi/agent/agent.py", line 577, in <listcomp>
    |     agent_data["images"] = [img.model_dump() for img in self.images]
    | AttributeError: 'dict' object has no attribute 'model_dump'
    +------------------------------------

@manthanguptaa
Copy link
Contributor

manthanguptaa commented Dec 11, 2024

Modelslab agent isn't giving me output and same with video agent

Screenshot 2024-12-11 at 5 10 45 PM

@manthanguptaa
Copy link
Contributor

manthanguptaa commented Dec 11, 2024

Tried running the FAL video agent on the playground and on the 2nd run I got this error. Also on the 2nd run the video player of the first output stops streaming with a gray infinite loading state

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/responses.py", line 259, in __call__
    await wrap(partial(self.listen_for_disconnect, receive))
  File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/responses.py", line 255, in wrap
    await func()
  File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/responses.py", line 232, in listen_for_disconnect
    message = await receive()
  File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/uvicorn/protocols/http/httptools_impl.py", line 555, in receive
    await self.message_event.wait()
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/asyncio/locks.py", line 226, in wait
    await fut
asyncio.exceptions.CancelledError: Cancelled by cancel scope 108e250d0

During handling of the above exception, another exception occurred:

  + Exception Group Traceback (most recent call last):
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/uvicorn/protocols/http/httptools_impl.py", line 401, in run_asgi
  |     result = await app(  # type: ignore[func-returns-value]
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
  |     return await self.app(scope, receive, send)
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/fastapi/applications.py", line 1054, in __call__
  |     await super().__call__(scope, receive, send)
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/applications.py", line 113, in __call__
  |     await self.middleware_stack(scope, receive, send)
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/middleware/errors.py", line 187, in __call__
  |     raise exc
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/middleware/errors.py", line 165, in __call__
  |     await self.app(scope, receive, _send)
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/middleware/cors.py", line 93, in __call__
  |     await self.simple_response(scope, receive, send, request_headers=headers)
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/middleware/cors.py", line 144, in simple_response
  |     await self.app(scope, receive, send)
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
  |     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
  |     raise exc
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
  |     await app(scope, receive, sender)
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/routing.py", line 715, in __call__
  |     await self.middleware_stack(scope, receive, send)
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/routing.py", line 735, in app
  |     await route.handle(scope, receive, send)
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/routing.py", line 288, in handle
  |     await self.app(scope, receive, send)
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/routing.py", line 76, in app
  |     await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
  |     raise exc
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
  |     await app(scope, receive, sender)
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/routing.py", line 74, in app
  |     await response(scope, receive, send)
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/responses.py", line 259, in __call__
  |     await wrap(partial(self.listen_for_disconnect, receive))
  |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 763, in __aexit__
  |     raise BaseExceptionGroup(
  | exceptiongroup.ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/responses.py", line 255, in wrap
    |     await func()
    |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/responses.py", line 244, in stream_response
    |     async for chunk in self.body_iterator:
    |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/concurrency.py", line 62, in iterate_in_threadpool
    |     yield await anyio.to_thread.run_sync(_next, as_iterator)
    |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/anyio/to_thread.py", line 56, in run_sync
    |     return await get_async_backend().run_sync_in_worker_thread(
    |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 2441, in run_sync_in_worker_thread
    |     return await future
    |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 943, in run
    |     result = context.run(func, *args)
    |   File "/Users/manthangupta/Desktop/lab/phidata/phienv/lib/python3.9/site-packages/starlette/concurrency.py", line 51, in _next
    |     return next(iterator)
    |   File "/Users/manthangupta/Desktop/lab/phidata/phi/playground/router.py", line 93, in chat_response_streamer
    |     for run_response_chunk in run_response:
    |   File "/Users/manthangupta/Desktop/lab/phidata/phi/agent/agent.py", line 1900, in _run
    |     self.write_to_storage()
    |   File "/Users/manthangupta/Desktop/lab/phidata/phi/agent/agent.py", line 739, in write_to_storage
    |     self._agent_session = self.storage.upsert(session=self.get_agent_session())
    |   File "/Users/manthangupta/Desktop/lab/phidata/phi/agent/agent.py", line 597, in get_agent_session
    |     agent_data=self.get_agent_data(),
    |   File "/Users/manthangupta/Desktop/lab/phidata/phi/agent/agent.py", line 579, in get_agent_data
    |     agent_data["videos"] = [vid.model_dump() for vid in self.videos]
    |   File "/Users/manthangupta/Desktop/lab/phidata/phi/agent/agent.py", line 579, in <listcomp>
    |     agent_data["videos"] = [vid.model_dump() for vid in self.videos]
    | AttributeError: 'dict' object has no attribute 'model_dump'
    +------------------------------------

Screenshot 2024-12-11 at 5 32 40 PM

anuragts and others added 8 commits December 11, 2024 07:27
* use-case-example-recipe-creator

* fixed linting

* fixed linting

* added changes

* directory name change

* fixes

* resolved comments

* resolved comments

* fixes

* fixes

* fix prompts and instructions

---------

Co-authored-by: Manthan Gupta <manthangupta109@gmail.com>
improved Description section for clarity
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure about making changes to assistants folder as we are trying to phase it out

],
markdown=True,
debug_mode=True,
add_history_to_messages=True,
add_datetime_to_instructions=True,
storage=SqlAgentStorage(table_name="image_agent", db_file="tmp/image_agent.db"),
storage=SqlAgentStorage(table_name="image_agent", db_file=image_agent_storage_file),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's keep it in tmp folder as we have it in .gitignore

name="Luma Video Agent",
agent_id="luma-video-agent",
llm=OpenAIChat(model="gpt-4o"),
tools=[LumaLabTools()], # Using the LumaLab tool we created
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unnecessary comment here

Comment on lines +12 to +13
import replicate
from replicate.helpers import FileOutput
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can add # type: ignore

@dirkbrnd dirkbrnd merged commit 7871732 into main Dec 12, 2024
1 check passed
@dirkbrnd dirkbrnd deleted the release/2.7.0 branch December 12, 2024 10:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants