Skip to content

Add stability ai image model #272

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 17 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
d15b69a
feat(Add-generate-support): Added first cut support for ultra, core a…
sayanc82 Jun 18, 2025
f371180
refactor(Modularize-code-for-better-readability): Added methods for c…
sayanc82 Jun 18, 2025
520a479
fix(stability): add type annotations and parameter validation
Jun 18, 2025
59e87d0
refactor(stability): separate error classes and add mode parameter
satsumas Jun 19, 2025
e93e660
refactor(stability): model_id is used by stream method to determine m…
satsumas Jun 19, 2025
461e756
refactor(stability): only SD3.5 accepts cfg_scale parameter
satsumas Jun 19, 2025
b5130b8
Merge branch 'sayan-stability-integration' into validate_schema
satsumas Jun 19, 2025
8ec7651
fix(stability): remove merge artefacts
satsumas Jun 19, 2025
a38c88e
feat(stability): StabilityAiImageModel raises exceptions from strands
satsumas Jun 20, 2025
8b9f411
fix(stability): Test image-to-image mode
satsumas Jun 20, 2025
a687845
Merge pull request #9 from Stability-AI/stability-implementation-comp…
sayanc82 Jun 21, 2025
3e13edf
feat(stability): response images can be returned as bytes
satsumas Jun 23, 2025
a18f184
Merge pull request #11 from Stability-AI/stability-implementation-com…
satsumas Jun 23, 2025
8a84f97
Merge pull request #12 from Stability-AI/sayan-stability-integration
satsumas Jun 23, 2025
0e90d6d
Merge branch 'main' into add-stability-ai-image-model
Jun 24, 2025
6c5628a
Merge pull request #1 from strands-agents/main
satsumas Jun 25, 2025
eb7f67d
Merge branch 'main' into add-stability-ai-image-model
satsumas Jun 25, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,3 +143,37 @@ If you discover a potential security issue in this project we ask that you notif
## Licensing

See the [LICENSE](./LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.

## Stability Specific Instructions

* Modified files

1. src/strands/event_loop/streaming.py
2. tests/strands/event_loop/test_streaming.py

These files have fixes to make the image return to agent consumers work.

* New Files
1. src/strands/models/_stabilityaiclient.py - A class with a rest client implementation for Stability
2. src/strands/models/stability.py - The main model provider implementation
3. tests-integ/test_model_stability.py - Integration test to test the image generation workflow.
4. tests/TODO.lis - A text file containing TODOs.
5. tests/strands/models/test_stability.py - Unit tests for the model provider


* Running the stability tests and testing the code

You can run the integration test for the stability model as follows
```
cd path/to/strands-agents-sdk-python
export STABILITY_API_KEY=<your api key>
hatch test tests-integ/test_model_stability.py
```

You can also install this as a editable module in a python project and write a agent

```
pip install -e /path/to/strands-agents-sdk-python
Sample code similar to
https://gist.github.com/sayanc82/17f0d2442acd78f74f8393f58b552c54
```
4 changes: 4 additions & 0 deletions TODO.lis
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Add support for all the parameters for core and 3.5
Do specific config validation for the 3 models, support parameters like cfg for sd3.5
Cleanup code :- Modularize. Move validation to its own methods for example.
Add documentation similar to other model providers
7 changes: 7 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -54,10 +54,12 @@ dev = [
"hatch>=1.0.0,<2.0.0",
"moto>=5.1.0,<6.0.0",
"mypy>=1.15.0,<2.0.0",
"Pillow>=11.0",
"pre-commit>=3.2.0,<4.2.0",
"pytest>=8.0.0,<9.0.0",
"pytest-asyncio>=0.26.0,<0.27.0",
"ruff>=0.4.4,<0.5.0",
"types-Pillow>=10.2.0.20240822",
]
docs = [
"sphinx>=5.0.0,<6.0.0",
Expand Down Expand Up @@ -119,6 +121,7 @@ lint-fix = [
features = ["anthropic", "litellm", "llamaapi", "ollama", "openai", "otel"]
extra-dependencies = [
"moto>=5.1.0,<6.0.0",
"Pillow>=11.0",
"pytest>=8.0.0,<9.0.0",
"pytest-asyncio>=0.26.0,<0.27.0",
"pytest-cov>=4.1.0,<5.0.0",
Expand Down Expand Up @@ -220,6 +223,10 @@ ignore_missing_imports = false
module = "litellm"
ignore_missing_imports = true

[[tool.mypy.overrides]]
module = "PIL"
ignore_missing_imports = true

[tool.ruff]
line-length = 120
include = ["examples/**/*.py", "src/**/*.py", "tests/**/*.py", "tests-integ/**/*.py"]
Expand Down
16 changes: 12 additions & 4 deletions src/strands/event_loop/streaming.py
Original file line number Diff line number Diff line change
Expand Up @@ -102,13 +102,14 @@ def handle_content_block_start(event: ContentBlockStartEvent) -> dict[str, Any]:


def handle_content_block_delta(
event: ContentBlockDeltaEvent, state: dict[str, Any]
event: ContentBlockDeltaEvent, state: dict[str, Any], **kwargs: Any
) -> tuple[dict[str, Any], dict[str, Any]]:
"""Handles content block delta updates by appending text, tool input, or reasoning content to the state.

Args:
event: Delta event.
state: The current state of message processing.
**kwargs: Additional keyword arguments to pass to the callback handler.

Returns:
Updated state with appended text or tool input.
Expand Down Expand Up @@ -150,7 +151,11 @@ def handle_content_block_delta(
"delta": delta_content,
"reasoning": True,
}

elif "image" in delta_content:
# Handle the new ImageContent structure
image_content = delta_content["image"]
state["image"] = image_content
callback_event["callback"] = {"image": image_content, "delta": delta_content, **kwargs}
return state, callback_event


Expand All @@ -168,7 +173,7 @@ def handle_content_block_stop(state: dict[str, Any]) -> dict[str, Any]:
current_tool_use = state["current_tool_use"]
text = state["text"]
reasoning_text = state["reasoningText"]

image = state["image"]
if current_tool_use:
if "input" not in current_tool_use:
current_tool_use["input"] = ""
Expand All @@ -192,7 +197,6 @@ def handle_content_block_stop(state: dict[str, Any]) -> dict[str, Any]:
elif text:
content.append({"text": text})
state["text"] = ""

elif reasoning_text:
content.append(
{
Expand All @@ -205,6 +209,9 @@ def handle_content_block_stop(state: dict[str, Any]) -> dict[str, Any]:
}
)
state["reasoningText"] = ""
elif image:
content.append({"image": image})
state["image"] = ""

return state

Expand Down Expand Up @@ -272,6 +279,7 @@ def process_stream(
"current_tool_use": {},
"reasoningText": "",
"signature": "",
"image": None,
}
state["content"] = state["message"]["content"]

Expand Down
Loading