models - bedrock - threading

[pgrayy]

Description

While we wait for an async bedrock client, we will execute converse_stream in a separate thread so as to prevent hogging on the async event loop.

Related Issues

#83

Documentation PR

This is an internal only change.

Type of Change

• Bug fix
• New feature
• Breaking change
• Documentation update
• Other (please describe):

Testing

How have you tested the change? Verify that the changes do not break functionality or introduce warnings in consuming repositories: agents-docs, agents-tools, agents-cli

• I ran hatch run prepare
• Ran the integration tests
• Ran the following test script:

import asyncio

from strands import Agent
from strands.models.bedrock import BedrockModel

prompt = "What is 2+2? Think through the steps."


async def func_a():
    agent = Agent(model=BedrockModel(), callback_handler=None)
    result = await agent.invoke_async(prompt)
    print(f"FUNC_A: {result.message}")


async def func_b():
    agent = Agent(model=BedrockModel(), callback_handler=None)
    result = await agent.invoke_async(prompt)
    print(f"FUNC_B: {result.message}")


async def func_c():
    agent = Agent(model=BedrockModel(), callback_handler=None)
    result = await agent.invoke_async(prompt)
    print(f"FUNC_C: {result.message}")


async def func_d():
    agent = Agent(model=BedrockModel(), callback_handler=None)
    result = await agent.invoke_async(prompt)
    print(f"FUNC_D: {result.message}")


async def main():
    await asyncio.gather(func_a(), func_b(), func_c(), func_d())


asyncio.run(main())

Without running converse_stream in a thread, all functions execute sequentially for a total average time of 10s. With threading, the functions execute concurrently and complete in an average of 4s.

Checklist

• I have read the CONTRIBUTING document
• I have added any necessary tests that prove my fix is effective or my feature works
• I have updated the documentation accordingly
• I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
• My changes generate no new warnings
• Any dependent changes have been merged and published

---

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.