Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(migration): Migrate reports with appropriate default checker ID #4191

Merged

Conversation

whisperity
Copy link
Contributor

Follows up #4089.

The first row in a table with an AUTOINCREMENT ID has the value 1, not 0. This was synchronised in the migration script to have server_default="1" such that all default constructed report rows have a valid FOREIGN KEY to the checkers table (the __FAKE__ checker). But the hand-written upgrade query filtered for 0 to get a batch of "unhandled rows", essentially preventing appropriate migration.

@whisperity whisperity added database 🗄️ Issues related to the database schema. bugfix 🔨 labels Mar 12, 2024
@whisperity whisperity added this to the release 6.24.0 milestone Mar 12, 2024
@whisperity whisperity force-pushed the feat/show-zero-checkers/fix-migration branch 3 times, most recently from 93b11da to 47851be Compare March 12, 2024 12:45
@cservakt
Copy link
Collaborator

I noticed that instead of utilizing SQLAlchemy's built in methods, raw SQL queries were executed directly. While this approach can work, it may introduce potential risks (for example when table or column name is changed).

Would it be possible to refactor the queries?

@whisperity
Copy link
Contributor Author

I noticed that instead of utilizing SQLAlchemy's built in methods, raw SQL queries were executed directly.

Would it be possible to refactor the queries?

That is very common during migration: [1], [2], [3]. When this script is executing, the schema in the database is not available in an ORM-y way, and there is a very high chance that the schema in the database "right now" does not match what the db model definitions in the Python package contain.

automap(reflect allows for some very limited functionality to figure out some things and go back to relying on SQLAlchemy to generate queries.

While this approach can work, it may introduce potential risks (for example when table or column name is changed).

Not even automap solves this. You have to rely on the exact table and column name, hence the

Report = Base.classes.reports  # 'reports' is the table name!
Checker = Base.classes.checkers

in the earlier code.

I can try changing this specific query to use automap here, as the Checkers table (or its contents) is not mutated in this function.

The first row in a table with an `AUTOINCREMENT` ID has the value 1, not
0. This was synchronised in the migration script to have
`server_default="1"` such that all default constructed `report` rows
have a **valid** `FOREIGN KEY` to the `checkers` table (the `__FAKE__`
checker). But the hand-written upgrade query filtered for `0` to get
a batch of "unhandled rows", essentially preventing appropriate
migration.
@whisperity whisperity force-pushed the feat/show-zero-checkers/fix-migration branch from 47851be to 3cfb1c2 Compare March 13, 2024 16:25
Copy link
Collaborator

@cservakt cservakt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Collaborator

@vodorok vodorok left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@vodorok vodorok merged commit 51bffbf into Ericsson:master Mar 20, 2024
7 of 8 checks passed
@whisperity whisperity deleted the feat/show-zero-checkers/fix-migration branch March 22, 2024 15:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bugfix 🔨 database 🗄️ Issues related to the database schema.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants