Skip to content

fix: open libraries from v9.5.0-pr1 in newer versions #815

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Feb 24, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions docs/updates/schema_changes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Save Format Changes

This page outlines the various changes made the TagStudio save file format over time, sometimes referred to as the "database" or "database file".

## JSON

| First Used | Last Used | Format | Location |
| ---------- | ----------------------------------------------------------------------- | ------ | --------------------------------------------- |
| v1.0.0 | [v9.4.2](https://github.com/TagStudioDev/TagStudio/releases/tag/v9.4.2) | JSON | `<Library Folder>`/.TagStudio/ts_library.json |

The legacy database format for public TagStudio releases [v9.1](https://github.com/TagStudioDev/TagStudio/tree/Alpha-v9.1) through [v9.4.2](https://github.com/TagStudioDev/TagStudio/releases/tag/v9.4.2). Variations of this format had been used privately since v1.0.0.

Replaced by the new SQLite format introduced in TagStudio [v9.5.0 Pre-Release 1](https://github.com/TagStudioDev/TagStudio/releases/tag/v9.5.0-pr1).

## DB_VERSION 6

| First Used | Last Used | Format | Location |
| ------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | ------ | ----------------------------------------------- |
| [v9.5.0-PR1](https://github.com/TagStudioDev/TagStudio/releases/tag/v9.5.0-pr1) | [v9.5.0-PR1](https://github.com/TagStudioDev/TagStudio/releases/tag/v9.5.0-pr1) | SQLite | `<Library Folder>`/.TagStudio/ts_library.sqlite |

The first public version of the SQLite save file format.

Migration from the legacy JSON format is provided via a walkthrough when opening a legacy library in TagStudio [v9.5.0 Pre-Release 1](https://github.com/TagStudioDev/TagStudio/releases/tag/v9.5.0-pr1) or later.

## DB_VERSION 7

| First Used | Last Used | Format | Location |
| ------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | ------ | ----------------------------------------------- |
| [v9.5.0-PR2](https://github.com/TagStudioDev/TagStudio/releases/tag/v9.5.0-pr2) | [v9.5.0-PR3](https://github.com/TagStudioDev/TagStudio/releases/tag/v9.5.0-pr3) | SQLite | `<Library Folder>`/.TagStudio/ts_library.sqlite |

### Changes

- Repairs "Description" fields to use a TEXT_LINE key instead of a TEXT_BOX key.
- Repairs tags that may have a disambiguation_id pointing towards a deleted tag.

## DB_VERSION 8

| First Used | Last Used | Format | Location |
| ------------------------------------------------------------------------------- | --------- | ------ | ----------------------------------------------- |
| [v9.5.0-PR4](https://github.com/TagStudioDev/TagStudio/releases/tag/v9.5.0-pr4) | _Current_ | SQLite | `<Library Folder>`/.TagStudio/ts_library.sqlite |

### Changes

- Adds the `color_border` column to `tag_colors` table. Used for instructing the [secondary color](../library/tag_color.md#secondary-color) to apply to a tag's border as a new optional behavior.
- Adds three new default colors: "Burgundy (TagStudio Shades)", "Dark Teal (TagStudio Shades)", and "Dark Lavender (TagStudio Shades)".
- Updates Neon colors to use the the new `color_border` property.
23 changes: 11 additions & 12 deletions tagstudio/src/core/library/alchemy/library.py
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,7 @@ class LibraryStatus:
success: bool
library_path: Path | None = None
message: str | None = None
msg_description: str | None = None
json_migration_req: bool = False


Expand Down Expand Up @@ -460,10 +461,12 @@ def open_sqlite_library(self, library_dir: Path, is_new: bool) -> LibraryStatus:

# Apply any post-SQL migration patches.
if not is_new:
if db_version < 8:
self.apply_db8_schema_changes(session)
if db_version == 6:
self.apply_db6_patches(session)
self.apply_repairs_for_db6(session)
if db_version >= 6 and db_version < 8:
self.apply_db7_patches(session)
self.apply_db8_default_data(session)

# Update DB_VERSION
if LibraryPrefs.DB_VERSION.default > db_version:
Expand All @@ -473,11 +476,8 @@ def open_sqlite_library(self, library_dir: Path, is_new: bool) -> LibraryStatus:
self.library_dir = library_dir
return LibraryStatus(success=True, library_path=library_dir)

def apply_db6_patches(self, session: Session):
"""Apply migration patches to a library with DB_VERSION 6.

DB_VERSION 6 was only used in v9.5.0-pr1.
"""
def apply_repairs_for_db6(self, session: Session):
"""Apply database repairs introduced in DB_VERSION 7."""
logger.info("[Library][Migration] Applying patches to DB_VERSION: 6 library...")
with session:
# Repair "Description" fields with a TEXT_LINE key instead of a TEXT_BOX key.
Expand All @@ -501,11 +501,8 @@ def apply_db6_patches(self, session: Session):

session.commit()

def apply_db7_patches(self, session: Session):
"""Apply migration patches to a library with DB_VERSION 7 or earlier.

DB_VERSION 7 was used from v9.5.0-pr2 to v9.5.0-pr3.
"""
def apply_db8_schema_changes(self, session: Session):
"""Apply database schema changes introduced in DB_VERSION 8."""
# TODO: Use Alembic for this part instead
# Add the missing color_border column to the TagColorGroups table.
color_border_stmt = text(
Expand All @@ -522,6 +519,8 @@ def apply_db7_patches(self, session: Session):
)
session.rollback()

def apply_db8_default_data(self, session: Session):
"""Apply default data changes introduced in DB_VERSION 8."""
tag_colors: list[TagColorGroup] = default_color_groups.standard()
tag_colors += default_color_groups.pastels()
tag_colors += default_color_groups.shades()
Expand Down
22 changes: 14 additions & 8 deletions tagstudio/src/qt/ts_qt.py
Original file line number Diff line number Diff line change
Expand Up @@ -710,14 +710,16 @@ def create_about_modal():
app.exec()
self.shutdown()

def show_error_message(self, message: str):
self.main_window.statusbar.showMessage(message, Qt.AlignmentFlag.AlignLeft)
self.main_window.landing_widget.set_status_label(message)
self.main_window.setWindowTitle(message)
def show_error_message(self, error_name: str, error_desc: str | None = None):
self.main_window.statusbar.showMessage(error_name, Qt.AlignmentFlag.AlignLeft)
self.main_window.landing_widget.set_status_label(error_name)
self.main_window.setWindowTitle(f"{self.base_title} - {error_name}")

msg_box = QMessageBox()
msg_box.setIcon(QMessageBox.Icon.Critical)
msg_box.setText(message)
msg_box.setText(error_name)
if error_desc:
msg_box.setInformativeText(error_desc)
msg_box.setWindowTitle(Translations["window.title.error"])
msg_box.addButton(Translations["generic.close"], QMessageBox.ButtonRole.AcceptRole)

Expand Down Expand Up @@ -1871,12 +1873,14 @@ def open_library(self, path: Path) -> None:
if self.lib.library_dir:
self.close_library()

open_status: LibraryStatus = None
open_status: LibraryStatus | None = None
try:
open_status = self.lib.open_library(path)
except Exception as e:
logger.exception(e)
open_status = LibraryStatus(success=False, library_path=path, message=type(e).__name__)
open_status = LibraryStatus(
success=False, library_path=path, message=type(e).__name__, msg_description=str(e)
)

# Migration is required
if open_status.json_migration_req:
Expand All @@ -1892,7 +1896,9 @@ def open_library(self, path: Path) -> None:
def init_library(self, path: Path, open_status: LibraryStatus):
if not open_status.success:
self.show_error_message(
open_status.message or Translations["window.message.error_opening_library"]
error_name=open_status.message
or Translations["window.message.error_opening_library"],
error_desc=open_status.msg_description,
)
return open_status

Expand Down
2 changes: 1 addition & 1 deletion tagstudio/src/qt/widgets/landing.py
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,7 @@ def animate_logo_pop(self):
# self.status_pos_anim.setEndValue(self.status_label.pos())
# self.status_pos_anim.start()

def set_status_label(self, text=str):
def set_status_label(self, text: str):
"""Set the text of the status label.

Args:
Expand Down
Binary file not shown.
Binary file not shown.
Binary file not shown.
47 changes: 47 additions & 0 deletions tagstudio/tests/test_db_migrations.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Copyright (C) 2025 Travis Abendshien (CyanVoxel).
# Licensed under the GPL-3.0 License.
# Created for TagStudio: https://github.com/CyanVoxel/TagStudio

import shutil
from pathlib import Path

import pytest
from src.core.constants import TS_FOLDER_NAME
from src.core.library.alchemy.library import Library

CWD = Path(__file__)
FIXTURES = "fixtures"
EMPTY_LIBRARIES = "empty_libraries"


@pytest.mark.parametrize(
"path",
[
str(Path(CWD.parent / FIXTURES / EMPTY_LIBRARIES / "DB_VERSION_6")),
str(Path(CWD.parent / FIXTURES / EMPTY_LIBRARIES / "DB_VERSION_7")),
str(Path(CWD.parent / FIXTURES / EMPTY_LIBRARIES / "DB_VERSION_8")),
],
)
def test_library_migrations(path: str):
library = Library()

# Copy libraries to temp dir so modifications don't show up in version control
original_path = Path(path)
temp_path = Path(CWD.parent / FIXTURES / EMPTY_LIBRARIES / "DB_VERSION_TEMP")
temp_path.mkdir(exist_ok=True)
temp_path_ts = temp_path / TS_FOLDER_NAME
temp_path_ts.mkdir(exist_ok=True)
shutil.copy(
original_path / TS_FOLDER_NAME / Library.SQL_FILENAME,
temp_path / TS_FOLDER_NAME / Library.SQL_FILENAME,
)

try:
status = library.open_library(library_dir=temp_path)
library.close()
shutil.rmtree(temp_path)
assert status.success
except Exception as e:
library.close()
shutil.rmtree(temp_path)
raise (e)