-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Labels
mindtrace-registryIssues raised from registry module in mindtrace packageIssues raised from registry module in mindtrace package
Milestone
Description
Description
Refactor the RegistryBackend API to do the following:
- Support batch operations;
- Move all locking operations to be internal to each backend;
- Remove
overwriteas part of the public API;
Explicitly, the RegistryBackend API should be:
# Objects API
push(self, name: str | list[str], version: str | list[str], local_path: str | Path | list[str | Path], metadata: dict | list[dict] | None = None)
pull(self, name: str | list[str], version: str | list[str], local_path: str | Path | list[str | Path])
delete(self, name: str | list[str], version: str | list[str])
# Object Metadata API
save_metadata(self, name: str | list[str], version: str | list[str], metadata: dict | list[dict])
fetch_metadata(self, name: str | list[str], version: str | list[str]) -> dict[str, dict]
delete_metadata(self, name: str | list[str], version: str | list[str])
# Serialization
register_materializer(self, object_class: str | list[str], materializer_class: str | list[str])
registered_materializers(self, object_class: str | None | list[str]) -> dict[str, str]
# Registry-level Metadata
save_registry_metadata(self, metadata: dict)
fetch_registry_metadata(self) -> dict
# Discovery
list_objects(self) -> list[str]
list_versions(self, name: str | list[str]) -> dict[str, list[str]]
has_object(self, name: str | list[str], version: str | list[str]) -> dict[str, bool]
# Utility Methods
validate_object_name(self, name: str | list[str]) -> None # raise exception if any names failThe associated Registry methods should also support batch operations:
save(self,
name: str | list[str],
obj: Any | list[Any],
*,
materializer: Type[BaseMaterializer] | None | list[Type[BaseMaterializer]] | list[None] = None,
version: str | list[str] | None = None,
init_params: Dict[str, Any] | None | list[Dict[str, Any] | None] = None,
metadata: Dict[str, Any] | None | list[Dict[str, Any] | None] = None,
)
# similarly for load, delete, info, etc.Acceptance Criteria
- Complete Registry implementation
- Complete support for the three main backends: Local, Minio and GCP
- 100% unit test coverage
- Expanded integration test suite
Priority
High (blocking release or critical path)
Estimated Effort (Story Points)
8 (1 week or more)
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
mindtrace-registryIssues raised from registry module in mindtrace packageIssues raised from registry module in mindtrace package