Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ Add issuer/verifier image_url to trustregistry #983

Merged
merged 56 commits into from
Oct 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
09912aa
add alembic to TR
cl0ete Sep 26, 2024
dfc0bf4
run alembic init
cl0ete Sep 26, 2024
2755814
create initial migration
cl0ete Sep 26, 2024
10b42cc
:art:
cl0ete Sep 26, 2024
b6670b2
alembic creates the tables now
cl0ete Sep 26, 2024
ae29e38
# pylint: disable=redefined-outer-name
cl0ete Sep 26, 2024
6396e69
:art:
cl0ete Sep 26, 2024
cb307a5
stamp the db
cl0ete Sep 27, 2024
904b43c
:art:
cl0ete Sep 27, 2024
e0e4aa6
poetry lock
cl0ete Oct 9, 2024
842bb1e
update test for new alembic code
cl0ete Oct 9, 2024
389d1a1
remove unused
cl0ete Oct 9, 2024
0b7172d
:art:
cl0ete Oct 9, 2024
df1cbbd
re-add mock_engine to tests
cl0ete Oct 9, 2024
51dab68
# pylint: disable=unused-argument
cl0ete Oct 9, 2024
1ba3605
:art:
cl0ete Oct 9, 2024
3a1a043
poetry lock
cl0ete Oct 11, 2024
89f7133
remove url from logs
cl0ete Oct 11, 2024
df5d789
fix db stamping logic
cl0ete Oct 11, 2024
3c968ac
:art:
cl0ete Oct 11, 2024
862af31
update tests for main
cl0ete Oct 11, 2024
41d01f9
# pylint: disable=W0718
cl0ete Oct 11, 2024
1f14705
:art:
cl0ete Oct 11, 2024
be47c7d
stamp the db
cl0ete Sep 27, 2024
98a35a8
:art:
cl0ete Sep 27, 2024
ebeb2fa
add image_url to register_actor call
cl0ete Aug 15, 2024
0559ba0
add image_url to update tenant logic
cl0ete Aug 15, 2024
3d2b8f9
update/fix tests
cl0ete Aug 15, 2024
e797538
add image_url to actor model
cl0ete Aug 15, 2024
ba2d1d3
add image_url to update actor crud
cl0ete Aug 15, 2024
a1fc83c
add image_url to db actor model
cl0ete Aug 15, 2024
4aef63d
:art:
cl0ete Aug 15, 2024
c666ac6
test new url is on TR with tenant update
cl0ete Aug 15, 2024
9ac549d
:art:
cl0ete Aug 16, 2024
26641fa
:art:
cl0ete Aug 16, 2024
68dad64
add alembic to trustregistry
cl0ete Aug 21, 2024
894041c
generate migration
cl0ete Aug 21, 2024
2896e63
expose trustregistry db to localhost
cl0ete Aug 21, 2024
fa2f1da
:art:
cl0ete Aug 21, 2024
b51370a
:art: import sort
cl0ete Aug 21, 2024
dc33ff4
:art:
cl0ete Sep 26, 2024
8cb83b7
regenerate migration script for image_url
cl0ete Sep 26, 2024
48a6c35
:art:
cl0ete Sep 26, 2024
3b95a1a
poetry lock
cl0ete Oct 11, 2024
0fe3363
clean up logging
cl0ete Oct 11, 2024
1d02461
# pylint: disable=redefined-outer-name
cl0ete Oct 11, 2024
52f5adf
remove unused import
cl0ete Oct 11, 2024
0df7ad6
:art:
cl0ete Oct 11, 2024
2a8a56f
fix migration logic for empty db
cl0ete Oct 11, 2024
50a99e7
update tests
cl0ete Oct 11, 2024
87e1a41
:art:
cl0ete Oct 11, 2024
dcadf8a
update readme
cl0ete Oct 14, 2024
4ebbe15
:art:
cl0ete Oct 14, 2024
f31be45
rename engine in check_migrations
cl0ete Oct 14, 2024
0c550be
remove alembic from dev-dependencies
cl0ete Oct 14, 2024
f14082e
:art:
cl0ete Oct 14, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions app/routes/admin/tenants.py
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,7 @@ async def create_tenant(
roles=roles,
did=onboard_result.did,
didcomm_invitation=onboard_result.didcomm_invitation,
image_url=body.image_url,
)
)
except HTTPException as http_error:
Expand Down
7 changes: 6 additions & 1 deletion app/services/onboarding/tenants.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ async def handle_tenant_update(

new_roles = update_request.roles or []
new_label = update_request.wallet_label
new_image_url = update_request.image_url

# See if this wallet belongs to an actor
actor = await fetch_actor_by_id(wallet_id)
Expand All @@ -48,9 +49,10 @@ async def handle_tenant_update(

if actor:
existing_roles = actor.roles
existing_image_url = actor.image_url
added_roles = list(set(new_roles) - set(existing_roles))

if new_label or added_roles: # Only update actor if
if new_label or added_roles or new_image_url: # Only update actor if
update_dict = {}
if new_label:
update_dict["name"] = new_label
Expand All @@ -76,6 +78,9 @@ async def handle_tenant_update(
update_dict["did"] = onboard_result.did
update_dict["didcomm_invitation"] = onboard_result.didcomm_invitation

if new_image_url and new_image_url != existing_image_url:
update_dict["image_url"] = new_image_url

updated_actor = actor.model_copy(update=update_dict)

await update_actor(updated_actor)
Expand Down
1 change: 1 addition & 0 deletions app/tests/e2e/test_tenants.py
Original file line number Diff line number Diff line change
Expand Up @@ -448,6 +448,7 @@ async def test_update_tenant_verifier_to_issuer(
assert_that(new_actor).has_name(new_wallet_label)
assert_that(new_actor).has_did(new_actor.did)
assert_that(new_actor.roles).contains_only("issuer", "verifier")
assert_that(new_actor.image_url).is_equal_to(new_image_url)

assert new_actor.didcomm_invitation is not None
finally:
Expand Down
1 change: 1 addition & 0 deletions app/tests/routes/admin_tenants/test_create_tenant.py
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,7 @@ async def test_create_tenant_success(roles):
roles=roles,
did=did,
didcomm_invitation=didcomm_invitation,
image_url=create_tenant_body.image_url,
)
)

Expand Down
2 changes: 2 additions & 0 deletions docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,8 @@ services:
- POSTGRES_USER=trustregistry
- POSTGRES_PASSWORD=trustregistry
- PGUSER=trustregistry
ports:
- 0.0.0.0:5432:5432
healthcheck:
test: ["CMD-SHELL", "pg_isready", "-d", "${POSTGRES_DB}"]
interval: 10s
Expand Down
1 change: 1 addition & 0 deletions shared/models/trustregistry.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ class Actor(BaseModel):
roles: List[TrustRegistryRole]
did: str
didcomm_invitation: Optional[str] = None
image_url: Optional[str] = None

@field_validator("did")
@classmethod
Expand Down
117 changes: 117 additions & 0 deletions trustregistry/alembic.ini
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
# A generic, single database configuration.

[alembic]
# path to migration scripts
# Use forward slashes (/) also on windows to provide an os agnostic path
script_location = migrations

# template used to generate migration file names; The default value is %%(rev)s_%%(slug)s
# Uncomment the line below if you want the files to be prepended with date and time
# see https://alembic.sqlalchemy.org/en/latest/tutorial.html#editing-the-ini-file
# for all available tokens
# file_template = %%(year)d_%%(month).2d_%%(day).2d_%%(hour).2d%%(minute).2d-%%(rev)s_%%(slug)s

# sys.path path, will be prepended to sys.path if present.
# defaults to the current working directory.
prepend_sys_path = ../

# timezone to use when rendering the date within the migration file
# as well as the filename.
# If specified, requires the python>=3.9 or backports.zoneinfo library.
# Any required deps can installed by adding `alembic[tz]` to the pip requirements
# string value is passed to ZoneInfo()
# leave blank for localtime
# timezone =

# max length of characters to apply to the "slug" field
# truncate_slug_length = 40

# set to 'true' to run the environment during
# the 'revision' command, regardless of autogenerate
# revision_environment = false

# set to 'true' to allow .pyc and .pyo files without
# a source .py file to be detected as revisions in the
# versions/ directory
# sourceless = false

# version location specification; This defaults
# to migrations/versions. When using multiple version
# directories, initial revisions must be specified with --version-path.
# The path separator used here should be the separator specified by "version_path_separator" below.
# version_locations = %(here)s/bar:%(here)s/bat:migrations/versions

# version path separator; As mentioned above, this is the character used to split
# version_locations. The default within new alembic.ini files is "os", which uses os.pathsep.
# If this key is omitted entirely, it falls back to the legacy behavior of splitting on spaces and/or commas.
# Valid values for version_path_separator are:
#
# version_path_separator = :
# version_path_separator = ;
# version_path_separator = space
# version_path_separator = newline
version_path_separator = os # Use os.pathsep. Default configuration used for new projects.

# set to 'true' to search source files recursively
# in each "version_locations" directory
# new in Alembic version 1.10
# recursive_version_locations = false

# the output encoding used when revision files
# are written from script.py.mako
# output_encoding = utf-8

sqlalchemy.url = postgresql://trustregistry:trustregistry@0.0.0.0:5432/trustregistry


[post_write_hooks]
# post_write_hooks defines scripts or Python functions that are run
# on newly generated revision scripts. See the documentation for further
# detail and examples

# format using "black" - use the console_scripts runner, against the "black" entrypoint
# hooks = black
# black.type = console_scripts
# black.entrypoint = black
# black.options = -l 79 REVISION_SCRIPT_FILENAME

# lint with attempts to fix using "ruff" - use the exec runner, execute a binary
# hooks = ruff
# ruff.type = exec
# ruff.executable = %(here)s/.venv/bin/ruff
# ruff.options = --fix REVISION_SCRIPT_FILENAME

# Logging configuration
[loggers]
keys = root,sqlalchemy,alembic

[handlers]
keys = console

[formatters]
keys = generic

[logger_root]
level = WARN
handlers = console
qualname =

[logger_sqlalchemy]
level = WARN
handlers =
qualname = sqlalchemy.engine

[logger_alembic]
level = INFO
handlers =
qualname = alembic

[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic

[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %H:%M:%S
1 change: 1 addition & 0 deletions trustregistry/crud.py
Original file line number Diff line number Diff line change
Expand Up @@ -181,6 +181,7 @@ def update_actor(db_session: Session, actor: Actor) -> db.Actor:
roles=actor.roles,
didcomm_invitation=actor.didcomm_invitation,
did=actor.did,
image_url=actor.image_url if actor.image_url else db_actor.image_url,
)
.returning(db.Actor)
)
Expand Down
1 change: 1 addition & 0 deletions trustregistry/db.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ class Actor(Base):
String, unique=True, index=True
)
did: Mapped[str] = mapped_column(String, unique=True, index=True)
image_url: Mapped[Optional[str]] = mapped_column(String, index=True)


class Schema(Base):
Expand Down
58 changes: 55 additions & 3 deletions trustregistry/main.py
Original file line number Diff line number Diff line change
@@ -1,14 +1,19 @@
import os
from contextlib import asynccontextmanager

from alembic import command
from alembic.config import Config
from alembic.migration import MigrationContext
from alembic.script import ScriptDirectory
from fastapi import Depends, FastAPI
from scalar_fastapi import get_scalar_api_reference
from sqlalchemy import inspect
from sqlalchemy.engine import Engine
from sqlalchemy.orm import Session

from shared.constants import PROJECT_VERSION
from shared.log_config import get_logger
from trustregistry import crud, db
from trustregistry import crud
from trustregistry.database import engine
from trustregistry.db import get_db
from trustregistry.registry import registry_actors, registry_schemas
Expand All @@ -19,10 +24,57 @@
ROOT_PATH = os.getenv("ROOT_PATH", "")


def check_migrations(db_engine: Engine, alembic_cfg: Config) -> bool:
# Check if alembic_version table exists
with db_engine.connect() as connection:
inspector = inspect(connection)
table_names = inspector.get_table_names()
has_alembic_version = "alembic_version" in table_names
has_actors_table = "actors" in table_names

script = ScriptDirectory.from_config(alembic_cfg)
if not has_alembic_version and has_actors_table:
logger.info(
"Alembic version table not found. Stamping with initial revision..."
)
try:
initial_revision = script.get_base()
command.stamp(alembic_cfg, initial_revision)
logger.info(
"Database stamped with initial migration version: {}", initial_revision
)
except Exception: # pylint: disable=W0718
logger.exception("Error stamping database")
raise

elif not has_alembic_version:
logger.info("Alembic version table not found.")
return False

# Get current revision
with db_engine.connect() as connection:
context = MigrationContext.configure(connection)
current_rev = context.get_current_revision()

head_rev = script.get_current_head()

return current_rev == head_rev


@asynccontextmanager
async def lifespan(_: FastAPI):
db.Base.metadata.create_all(bind=engine)
engine.dispose()
alembic_cfg = Config("alembic.ini")

if not check_migrations(engine, alembic_cfg):
logger.info("Applying database migrations...")
try:
command.upgrade(alembic_cfg, "head")
logger.info("Database schema is up to date.")
except Exception: # pylint: disable=broad-except
logger.exception("Error during migration")
raise
else:
logger.info("Database is up to date. No migrations needed.")

logger.debug("TrustRegistry startup: Validate tables are created")
with engine.connect() as connection:
Expand Down
15 changes: 15 additions & 0 deletions trustregistry/migrations/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Alembic Migrations

## Overview

This folder contains the migration scripts for the trust registry database.
These scripts are used to manage changes to the database schema in a consistent and version-controlled manner.

### Files and Directories

- `env.py`: This is the configuration file for Alembic, the database migration tool.
It sets up the database connection and other settings required for migrations.
- `script.py.mako`: This is a template file used by Alembic to generate new migration scripts.
It contains placeholders for the migration logic.
- `versions/`: This directory contains the individual migration scripts.
Each script is named with a unique identifier and describes a specific change to the database schema.
Loading
Loading