-
Notifications
You must be signed in to change notification settings - Fork 12
Fix postgres migrations #433
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,112 @@ | ||
| # Postgres integrations | ||
|
|
||
| Draive ships with Postgres-backed implementations for the most common persistence interfaces so you can plug relational storage into your workflows without writing adapters. All helpers live in `draive.postgres` and reuse the shared `haiway.postgres.Postgres` connection states. | ||
|
|
||
| ## Bootstrapping the Postgres context | ||
|
|
||
| Before using any adapter ensure a connection pool is available inside your context scope. The helpers lean on `PostgresConnectionPool` and the `Postgres` facade exported from `draive.postgres`. | ||
|
|
||
| ```python | ||
| from draive import ctx | ||
| from draive.postgres import ( | ||
| Postgres, | ||
| PostgresConnectionPool, | ||
| PostgresConfigurationRepository, | ||
| PostgresInstructionsRepository, | ||
| PostgresModelMemory, | ||
| ) | ||
|
|
||
| async with ctx.scope( | ||
| "postgres-demo", | ||
| PostgresConfigurationRepository(), # use use postgres configurations | ||
| PostgresInstructionsRepository(), # use postgres instructions | ||
| disposables=( | ||
| PostgresConnectionPool.of(dsn="postgresql://draive:secret@localhost:5432/draive"), | ||
| ), | ||
| ): | ||
| session_memory = PostgresModelMemory("demo-session") | ||
| ``` | ||
|
|
||
| Each adapter relies on the same connection scope, so you can freely mix them within a single context. | ||
|
|
||
| ## ConfigurationRepository implementation | ||
|
|
||
| `PostgresConfigurationRepository` persists configuration snapshots inside a `configurations` table and keeps a bounded LRU cache to avoid repeated fetches. The table must expose the schema used in the implementation: | ||
|
|
||
| ```sql | ||
| CREATE TABLE configurations ( | ||
| identifier TEXT NOT NULL, | ||
| content JSONB NOT NULL, | ||
| created TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP, | ||
| PRIMARY KEY (identifier, created) | ||
| ); | ||
| ``` | ||
|
|
||
| Key capabilities: | ||
| - `configurations()` returns every known identifier using cached results (limit 1, default 10 minute TTL). | ||
| - `load(config, identifier)` fetches the newest JSON document per identifier and parses it into a requested configuration type. | ||
| - `load_raw(identifier)` fetches raw Mapping for given identifier. | ||
| - `define(config)` upserts a new configuration snapshot and clears both caches, guaranteeing fresh reads on the next call. | ||
| - `remove(identifier)` deletes all historical snapshots for the identifier and purges caches. | ||
|
|
||
| Tune memory pressure through `cache_limit` and `cache_expiration` arguments when instantiating the repository. | ||
|
|
||
| ## InstructionsRepository implementation | ||
|
|
||
| `PostgresInstructionsRepository` mirrors the behaviour of the in-memory instructions repository while persisting values in a dedicated `instructions` table: | ||
|
|
||
| ```sql | ||
| CREATE TABLE instructions ( | ||
| name TEXT NOT NULL, | ||
| description TEXT DEFAULT NULL, | ||
| content TEXT NOT NULL, | ||
| arguments JSONB NOT NULL DEFAULT '[]'::jsonb, | ||
| meta JSONB NOT NULL DEFAULT '{}'::jsonb, | ||
| created TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP, | ||
| PRIMARY KEY (name, created) | ||
| ); | ||
| ``` | ||
|
|
||
| Highlights: | ||
| - `available_instructions()` returns structured `InstructionsDeclaration` objects with cached results for quick catalog views. | ||
| - `resolve(instructions, arguments)` resolves the latest instruction body, leveraging a dedicated cache keyed by name and utilizing the provided arguments. | ||
| - `load(instructions)` loads the raw latest instructions keyed by name. | ||
| - `define(instructions, content)` stores new revisions and invalidates caches so subsequent reads return the fresh version. | ||
| - `remove(instructions)` removes all revisions for the instruction and drops relevant cache entries. | ||
|
|
||
| This adapter is ideal when you author system prompts and tool manifests centrally and want version history per instruction. | ||
|
|
||
| ## ModelMemory implementation | ||
|
|
||
| `PostgresModelMemory` enables durable conversational memory by persisting variables and context elements in three tables sharing the same identifier: | ||
|
|
||
| ```sql | ||
| CREATE TABLE memories ( | ||
| identifier TEXT NOT NULL, | ||
| created TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP, | ||
| PRIMARY KEY (identifier) | ||
| ); | ||
|
|
||
| CREATE TABLE memories_variables ( | ||
| identifier TEXT NOT NULL REFERENCES memories (identifier) ON DELETE CASCADE, | ||
| variables JSONB NOT NULL DEFAULT '{}'::jsonb, | ||
| created TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP | ||
| ); | ||
|
|
||
| CREATE TABLE memories_elements ( | ||
| identifier TEXT NOT NULL REFERENCES memories (identifier) ON DELETE CASCADE, | ||
| content JSONB NOT NULL, | ||
| created TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP | ||
| ); | ||
| ``` | ||
|
|
||
| Capabilities: | ||
| - `recall(limit=...)` fetches the latest variables and replayable context elements (inputs/outputs) respecting the optional `recall_limit` supplied to the factory. | ||
| - `remember(*items, variables=...)` persists new context elements and optionally a fresh variable snapshot in a single transaction. | ||
| - `maintenance(variables=...)` ensures the base `memories` row exists and can seed default variables without appending messages. | ||
|
|
||
| Use the memory helper when you need stateful chat sessions, per-user progressive profiling, or auditable interaction logs. Set `recall_limit` to bound the amount of context loaded back into generation pipelines. | ||
|
|
||
| ## Putting it together | ||
|
|
||
| Combine these adapters with higher-level Draive components to centralise operational data in Postgres. For example, wire the configuration repository into your configuration state, keep reusable instruction sets shareable across teams, and persist model interactions for analytics—all while letting `haiway` manage connection pooling and logging through the active context. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -12,12 +12,10 @@ | |
| # CREATE TABLE configurations ( | ||
| # identifier TEXT NOT NULL, | ||
| # content JSONB NOT NULL, | ||
| # created TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP | ||
| # created TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP, | ||
| # PRIMARY KEY (identifier, created) | ||
|
Comment on lines
+15
to
+16
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🧩 Analysis chainVerify migration path for composite primary key change. The schema now uses a composite primary key
Run the following script to check for migration scripts and schema documentation: 🏁 Script executed: #!/bin/bash
# Description: Search for postgres migration scripts and schema setup documentation
# Look for migration scripts
fd -e sql -e py 'migrat' --exec echo "Found migration file: {}"
# Search for schema setup or migration documentation
rg -n "CREATE TABLE configurations|ALTER TABLE configurations" --type=sql --type=py
# Check if there are other references to the old schema
rg -n "PRIMARY KEY.*identifier[^,]" --type=sql --type=pyLength of output: 135 Provide a migration for the new composite primary key. No migration script exists to add the
🤖 Prompt for AI Agents |
||
| # ); | ||
| # | ||
| # CREATE INDEX configurations_identifier_created_idx | ||
| # ON configurations (identifier, created DESC); | ||
| # | ||
|
|
||
|
|
||
| def PostgresConfigurationRepository( | ||
|
|
@@ -38,6 +36,7 @@ def PostgresConfigurationRepository( | |
| ConfigurationRepository | ||
| Repository facade backed by the ``configurations`` Postgres table. | ||
| """ | ||
|
|
||
| @cache( | ||
| limit=1, | ||
| expiration=cache_expiration, | ||
|
|
@@ -49,7 +48,7 @@ async def listing( | |
| results: Sequence[PostgresRow] = await Postgres.fetch( | ||
| """ | ||
| SELECT DISTINCT ON (identifier) | ||
| identifier | ||
| identifier::TEXT | ||
|
|
||
| FROM | ||
| configurations | ||
|
|
@@ -75,8 +74,8 @@ async def loading( | |
| loaded: PostgresRow | None = await Postgres.fetch_one( | ||
| """ | ||
| SELECT DISTINCT ON (identifier) | ||
| identifier, | ||
| content | ||
| identifier::TEXT, | ||
| content::TEXT | ||
|
|
||
| FROM | ||
| configurations | ||
|
|
@@ -118,8 +117,8 @@ async def define( | |
| ) | ||
|
|
||
| VALUES ( | ||
| $1, | ||
| $2::jsonb | ||
| $1::TEXT, | ||
| $2::JSONB | ||
| ); | ||
| """, | ||
| identifier, | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧩 Analysis chain
Verify method name "configurations()".
The documentation refers to
configurations()as returning "every known identifier". Confirm this matches the actual method name inPostgresConfigurationRepository. Based on typical repository patterns, it might be named something likelist(),identifiers(), oravailable_configurations().🏁 Script executed:
Length of output: 25
Use the correct method name
listing()in documentationReplace
configurations()withlisting()in docs/guides/Postgres.md to match the actual method inPostgresConfigurationRepository.🤖 Prompt for AI Agents