Rework MediaContent

AGENTS.md

@@ -5,16 +5,45 @@ Rules for coding agents to contribute correctly and safely.
 ## Development Toolchain
 
 - Python: 3.12
-- Virtualenv: managed by uv, available at `./.venv`
-- Formatting: Ruff formatter (`make format`). No other formatter
+- Virtualenv: managed by uv, available at `./.venv`, assume already set up and working within venv
+- Formatting: Ruff formatter (`make format`), no other formatter
 - Linters/Type-checkers: Ruff, Bandit, Pyright (strict). Run via `make lint`
 - Tests: `make test` or targeted `pytest tests/test_...::test_...`
 
+## Framework Foundation (Haiway)
+
+- Draive is built on top of Haiway (state, context, observability, config). See: https://github.com/miquido/haiway
+- Import symbols from `haiway` directly: `from haiway import State, ctx`
+- Use context scoping (`ctx.scope(...)`) to bind active `State` instances and avoid global state
+- All logs go through `ctx.log_*`; do not use `print`.
+
+## Project Layout
+
+Top-level code lives under `src/draive/`, key packages and what they contain:
+
+- `draive/models/` — core model abstractions: `GenerativeModel`, tools (`models/tools`), instructions handling
+- `draive/generation/` — typed generation facades for text, image, audio, and model wiring (`state.py`, `types.py`, `default.py`)
+- `draive/conversation/` — higher-level chat/realtime conversations (completion and realtime sessions)
+- `draive/multimodal/` — content types and helpers: `MultimodalContent`, `TextContent`, `ArtifactContent`
+- `draive/resources/` — resource references and blobs: `ResourceContent`, `ResourceReference`, repository interfaces
+- `draive/parameters/` — strongly-typed parameter schemas and validation helpers
+- `draive/embedding/` — vector operations, similarity, indexing, and typed embedding states
+- `draive/guardrails/` — moderation, privacy, quality verification states and types
+- `draive/stages/` — pipeline stages abstractions and helpers
+- `draive/utils/` — utilities (e.g., `Memory`, `VectorIndex`)
+- Provider adapters — unified shape per provider:
+  - `draive/openai/`, `draive/anthropic/`, `draive/mistral/`, `draive/gemini/`, `draive/vllm/`, `draive/ollama/`, `draive/bedrock/`, `draive/cohere/`
+  - Each has `config.py`, `client.py`, `api.py`, and feature-specific modules.
+- Integrations: `draive/httpx/`, `draive/mcp/`, `draive/opentelemetry/` (opt-in extras)
+
+Public exports are centralized in `src/draive/__init__.py`.
+
 ## Style & Patterns
 
 ### Typing & Immutability
 
 - Strict typing only: no untyped public APIs, no loose `Any` unless required by third-party boundaries
+- Prefer explicit attribute access with static types. Avoid dynamic `getattr` except at narrow boundaries.
 - Prefer abstract immutable protocols: `Mapping`, `Sequence`, `Iterable` over `dict`/`list`/`set` in public types
 - Use `final` where applicable; avoid complex inheritance, prefer type composition
 - Use precise unions (`|`) and narrow with `match`/`isinstance`, avoid `cast` unless provably safe and localized
@@ -26,6 +55,50 @@ Rules for coding agents to contribute correctly and safely.
 - Access active state through `haiway.ctx` inside async scopes (`ctx.scope(...)`).
 - Public state methods that dispatch on the active instance should use `@statemethod` (see `GenerativeModel`).
 
+#### Examples:
+
+State with instantiation helper:
+```python
+from typing import Self
+from haiway import State, statemethod, Meta, MetaValues
+
+class Counter(State):
+    @classmethod
+    def of(
+        cls,
+        *,
+        start: int = 0,
+        meta: Meta | MetaValues | None = None,
+    ) -> Self:
+        return cls(
+            value=start,
+            meta=Meta.of(meta),
+        )
+
+    value: int
+    meta: Meta
+```
+
+State with statemethod helper:
+```python
+from typing import Self, Protocol
+from haiway import State, statemethod
+
+class Printing(Protocol):
+    async def __call__(self, text: str) -> None: ...
+
+class Printer(State):
+    @statemethod
+    async def print(
+        self,
+        *,
+        value: str | int | float,
+    ) -> None:
+        await self.printing(str(value))
+
+    printing: Printing
+```
+
 ### Observability & Logging
 
 - Use `ctx.log_debug/info/warn/error` for logs; do not use `print`
@@ -36,16 +109,19 @@ Rules for coding agents to contribute correctly and safely.
 ### Concurrency & Async
 
 - All I/O is async, keep boundaries async and use `ctx.spawn` for detached tasks
-- Rely on asyncio package and coroutines, avoid custom threading
+- Ensure structured concurrency concepts and valid coroutine usage
+- Rely on haiway and asyncio packages with coroutines, avoid custom threading
 
 ### Exceptions & Error Translation
 
 - Translate provider/SDK errors into appropriate typed exceptions
-- Don’t raise bare `Exception`, preserve provider/model identifiers in exception construction
+- Don’t raise bare `Exception`, preserve contextual information in exception construction
 
 ### Multimodal
 
-- Build content with `MultimodalContent.of(...)` and prefer composing content blocks explicitly.
+- Build content with `MultimodalContent.of(...)` and prefer composing content blocks explicitly
+- Use ResourceContent/Reference for media and data blobs
+- Wrap custom types and data within ArtifactContent, use hidden when needed
 
 ## Testing & CI
 
@@ -54,15 +130,50 @@ Rules for coding agents to contribute correctly and safely.
 - Use fixtures from `tests/` or add focused ones; avoid heavy integration scaffolding
 - Linting/type gates must be clean: `make format` then `make lint`
 
+### Async tests
+
+- Use `pytest-asyncio` for coroutine tests (`@pytest.mark.asyncio`).
+- Prefer scoping with `ctx.scope(...)` and bind required `State` instances explicitly.
+- Avoid real I/O and network; stub provider calls and HTTP.
+
+#### Examples
+
+```python
+import pytest
+from draive import State, ctx
+
+class Example(State):
+    name: str
+
+@pytest.mark.asyncio
+async def test_greeter_returns_greeting() -> None:
+    async with ctx.scope(Example(name="Ada")):
+        example: Example = ctx.state(Example)
+        assert example.name == "Ada"
+```
+
+### Self verification
+
+- Ensure type checking soundness as a part of the workflow
+- Do not mute or ignore errors, double check correctness and seek for solutions
+- Verify code correctness with unit tests or by running ad-hoc scripts
+- Ask for additional guidance and confirmation when uncertain or about to modify additional elements
+
 ## Documentation
 
 - Public symbols: add NumPy-style docstrings. Include Parameters/Returns/Raises sections and rationale when not obvious
 - Internal helpers: avoid docstrings, keep names self-explanatory
-- If behavior/API changes, update relevant docs under `docs/` and examples if applicable.
+- If behavior/API changes, update relevant docs under `docs/` and examples if applicable
+- Skip module docstrings
 
-## Public API & Deprecations
+### Docs (MkDocs)
 
-- Export new public types/functions via `src/draive/__init__.py` and relevant package `__init__.py`.
+- Site is built with MkDocs + Material and `mkdocstrings` for API docs.
+- Author pages under `docs/` and register navigation in `mkdocs.yml` (`nav:` section).
+- Preview locally: `make docs-server` (serves at http://127.0.0.1:8000).
+- Build static site: `make docs` (outputs to `site/`).
+- Keep docstrings high‑quality; `mkdocstrings` pulls them into reference pages.
+- When adding public APIs, update examples/guides as needed and ensure cross‑links render.
 
 ## Security & Secrets
 
@@ -77,11 +188,31 @@ Rules for coding agents to contribute correctly and safely.
 - Types: strict, no ignores, no loosening of typing
 - API surface: update `__init__.py` exports and docs if needed
 
-## Extras
+## Code Search (ast-grep)
+
+Use ast-grep for precise, structural search and refactors. Prefer `rg` for simple text matches.
+
+- Invocation: `sg run --lang python --pattern "<PATTERN>" [FLAGS]`
+- Useful patterns for this repo:
+  - `ctx.scope($X)` — find context scopes
+  - `@statemethod` — find state-dispatched classmethods
+  - `class $NAME(State)` — locate `haiway.State` subclasses
+  - `MultimodalContent.of($X)` — multimodal composition sites
+  - `$FUNC(meta=$META)` — functions receiving `meta`
+- Common flags:
+  - `--json` — machine-readable results
+  - `--selector field_declaration` — restrict matches (e.g., type fields)
+  - `--rewrite "<REWRITE>"` — propose changes; always review diffs first
+
+### Examples:
+
+```bash
+# Where do we open context scopes?
+sg run --lang python --pattern "ctx.scope($X)" src
 
-### Code Search (ast-grep)
+# Find all State types
+sg run --lang python --pattern "class $NAME(State)" src
 
-- Use for precise AST queries and code search; prefer `rg` for simple text search
-- Command: `sg run --lang python --pattern "<PATTERN>"`
-- Patterns: `Meta.of($VAR)`, `meta: Meta`, `$FUNC(meta=$META)`, `class $NAME(Meta)`
-- Flags: `--selector field_declaration`, `--json`, `--rewrite <FIX>` (review diffs)
+# Find multimodal builders
+sg run --lang python --pattern "MultimodalContent.of($X)" src
+```

Makefile

@@ -10,7 +10,7 @@ TESTS_PATH := tests
 -include .env
 
 ifndef UV_VERSION
-	UV_VERSION := 0.8.14
+	UV_VERSION := 0.8.17
 endif
 
 .PHONY: uv_check venv sync update format lint test docs docs-server release
@@ -22,7 +22,7 @@ uv_check:
 	# Install if not present
 	@if ! command -v uv > /dev/null; then \
 		echo '...installing uv...'; \
-		curl -fLsS https://astral.sh/uv/install.sh | sh -s -- --version $(UV_VERSION); \
+		curl -fLsS https://astral.sh/uv/install.sh | sh; \
 		if [ $$? -ne 0 ]; then \
 			echo "...installing uv failed!"; \
 			exit 1; \
@@ -34,7 +34,7 @@ uv_check:
 		CURRENT_VERSION=$$(uv --version | head -n1 | cut -d" " -f2); \
 		if [ "$$(printf "%s\n%s" "$(UV_VERSION)" "$$CURRENT_VERSION" | sort -V | head -n1)" != "$(UV_VERSION)" ]; then \
 			echo '...updating uv...'; \
-			curl -fLsS https://astral.sh/uv/install.sh | sh -s -- --version $(UV_VERSION); \
+			curl -fLsS https://astral.sh/uv/install.sh | sh; \
 			if [ $$? -ne 0 ]; then \
 				echo "...updating uv failed!"; \
 				exit 1; \
@@ -62,7 +62,7 @@ sync: uv_check
 	@echo '...finished!'
 
 # Update and lock dependencies from pyproject.toml
-update:
+update: uv_check
 	@echo '# Updating dependencies...'
 	@uv sync --all-groups --all-extras --upgrade
 	@echo '...finished!'

docs/cookbooks/BasicDataExtraction.md

@@ -6,12 +6,7 @@ One of the most useful abilities of LLMs is to extract information from an unstr
 ```python
 from draive import load_env
 
-load_env() # load .env variables
-```
-
-## Data source
 
-For our data source we will use a short text about John Doe. For more advanced use case you might need some more advanced text extraction techniques.
 
 
 ```python
@@ -28,12 +23,7 @@ John's days are filled with exploring the city's diverse neighborhoods, trying n
 Despite the occasional homesickness for his family and the familiarity of his Texas farm, John knows that Vancouver is where he belongs. The city has captured his heart, and he can't imagine living anywhere else. He dreams of one day working in the field of environmental conservation, helping to protect the natural wonders that made him fall in love with Canada.
 
 As John reflects on his journey from a small farm in Texas to the vibrant city of Vancouver, he feels a sense of pride and accomplishment. He knows that his seven-year-old self would be proud of the life he has built in the country that captured his imagination all those years ago. With a smile on his face, John looks forward to the future and all the adventures that Vancouver has in store for him.
-"""
-```
 
-## Data structure
-
-`DataModel` base class is a perfect tool to define required data structure. We will use it to define a simple structure for this example. Please see [AdvancedState](../guides/AdvancedState.md) for more details about the advanced data model customization.
 
 
 ```python
@@ -43,12 +33,7 @@ class PersonalData(DataModel):
     first_name: str
     last_name: str
     age: int | None = None
-    country: str | None = None
-```
 
-## Extraction
-
-Since we are going to generate a structured output we will use the `ModelGeneration.generate` helper which automatically decodes the result of a model call into a python object. We have to prepare the execution context, provide instructions, the source document and specify the data model, and we can expect extracted data to be available. Keep in mind that depending on the task it might be beneficial to prepare a specific instruction and/or specify more details about the model itself.
 
 
 ```python
@@ -76,12 +61,7 @@ async with ctx.scope(
     first_name: John
     last_name: Doe
     age: 21
-    country: Canada
-
-
-## Customization
 
-We can customize data extraction by specifying more details about the model itself i.e. description of some fields. We can also choose to take more control over the prompt used to generate the model. Instead of relying on the automatically generated model description we can specify it manually or provide a placeholder inside the instruction to put it in specific place. Additionally we can choose between full json schema description and the simplified description which works better with smaller, less capable models.
 
 
 ```python

docs/cookbooks/BasicRAG.md

@@ -6,12 +6,7 @@ RAG is a technique of adding additional information to the LLM context, to achie
 ```python
 from draive import load_env
 
-load_env() # load .env variables
-```
-
-## Data preparation
 
-We are going to use the same text about John Doe as we did in [Basic Data Extraction](./BasicDataExtraction.md) as our input. After assigning it to a variable, we will prepare it for embedding by splitting the original text into smaller chunks. This allows us to fit the data into limited context windows and filter out unnecessary information. We’ll use a basic splitter that looks for paragraph structure defined by multiple subsequent newlines in the text. Chunks will be put into a structured envelope that allows storing additional information such as the full document text or extra metadata.
 
 ```python
 from draive import DataModel, ctx, split_text
@@ -71,12 +66,7 @@ print("\n---\n".join(chunk.content for chunk in document_chunks))
     ---
     As John reflects on his journey from a small farm in Texas to the vibrant city of Vancouver, he feels a sense of pride and accomplishment. He knows that his seven-year-old self would be proud of the life he has built in the country that captured his imagination all those years ago. With a smile on his
     ---
-    that captured his imagination all those years ago. With a smile on his face, John looks forward to the future and all the adventures that Vancouver has in store for him.
-
-
-## Data indexing
 
-When we have prepared the data, we can now use a vector index to make it searchable. We are going to use in-memory, volatile vector index which can be useful for a quick search. Preparing the index requires defining the text embedding method. In this example we are going to use OpenAI embedding solution. After defining all required parameters and providers we can prepare the index using our data. In order to ensure proper data embedding it is required to specify what value will be used to prepare the vector. In this case we specify the chunk content to be used.
 
 
 ```python
@@ -102,12 +92,7 @@ async with ctx.scope(
         values=document_chunks,
         # define what value will be embedded for each chunk
         attribute=DocumentChunk._.content,
-    )
-```
-
-## Searching the index
 
-Now we have everything set up to do our search. Instead of searching directly, we are going to leverage the ability of LLMs to call tools. We have to define a search tool for that task, which will use our index to deliver the results. LLM will make use of this tool to look for the answer based on the search result.
 
 
 ```python