Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FR: Do not fail when some db migrations are already in place. #1797

Open
real-yfprojects opened this issue Sep 1, 2023 · 0 comments
Open
Labels
type:enhancement Improvement of an existing function

Comments

@real-yfprojects
Copy link
Collaborator

real-yfprojects commented Sep 1, 2023

The problem

As reported in #1761 and multiple previous issues sometimes database migrations seem to be only half way done. This leads to consequent error messages like

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/vorta/__main__.py", line 66, in main
    init_db(sqlite_db)
  File "/usr/lib/python3/dist-packages/vorta/store/connection.py", line 83, in init_db
    run_migrations(current_schema, con)
  File "/usr/lib/python3/dist-packages/vorta/store/migrations.py", line 190, in run_migrations
    _apply_schema_update(
  File "/usr/lib/python3/dist-packages/vorta/store/migrations.py", line 225, in _apply_schema_update
    migrate(*operations)
  File "/usr/lib/python3/dist-packages/playhouse/migrate.py", line 886, in migrate
    operation.run()
  File "/usr/lib/python3/dist-packages/peewee.py", line 3177, in execute_sql
    cursor.execute(sql, params or ())
sqlite3.OperationalError: duplicate column name: schedule_interval_unit

Requested Solution

Instead of failing vorta could only do those migrations actually needed by comparing the state of the database with the expected state. This will also help when testing PRs that add db migrations.

However this isn't straightforward at all as far as I can tell at the moment. These are possible roadblockers:

  • peewee doesn't provide such features and their might not be a viable third party library we can use together with peewee.
  • We sometimes have special data altering migrations steps like the following and I don't know how those could be implemented with the approach requested in this FR.
    if current_schema.version < 11:
    _apply_schema_update(current_schema, 11)
    for profile in BackupProfileModel:
    if profile.compression == 'zstd':
    profile.compression = 'zstd,3'
    if profile.compression == 'lzma,6':
    profile.compression = 'auto,lzma,6'
    profile.save()

Alternatives

  • Catch error and ignore it.
  • Find cause of incomplete migrations.
@real-yfprojects real-yfprojects added the type:enhancement Improvement of an existing function label Sep 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:enhancement Improvement of an existing function
Projects
None yet
Development

No branches or pull requests

1 participant