Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Move category creation and UUID update to a more performant data migration #3437

Merged
merged 3 commits into from
Jul 27, 2022
Merged

Move category creation and UUID update to a more performant data migration #3437

merged 3 commits into from
Jul 27, 2022

Conversation

mgdelacroix
Copy link
Contributor

Summary

This PR replaces the approach of creating the categories and category boards on a migration and updating their IDs on a data migration with one that directly generates that data in a data migration. This new approach inserts the data in batches of 1000 values at a time, which combined with the speed of the inserts makes the whole migration way faster.

@mgdelacroix mgdelacroix added 2: Dev Review Requires review by a core committer CherryPick/Approved Meant for the quality or patch release tracked in the milestone labels Jul 27, 2022
@mgdelacroix mgdelacroix added this to the v7.2 milestone Jul 27, 2022
@mgdelacroix mgdelacroix requested a review from a team as a code owner July 27, 2022 19:29
jespino
jespino approved these changes Jul 27, 2022
View reviewed changes
Copy link
Contributor

@jespino jespino left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

sbishel
sbishel approved these changes Jul 27, 2022
View reviewed changes
Copy link
Collaborator

@sbishel sbishel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add any logging, so we/admins know what point in the sequence we are at? Or is it performant enough that its unnecessary.

LGTM

server/services/store/sqlstore/data_migrations.go Show resolved Hide resolved
server/services/store/sqlstore/data_migrations.go Show resolved Hide resolved
@mgdelacroix mgdelacroix merged commit 6616a16 into main Jul 27, 2022
@mattermod
Copy link
Contributor

Cherry pick is scheduled.

@mgdelacroix mgdelacroix deleted the category-uuid-migration-optimization branch July 27, 2022 21:21
mattermost-build pushed a commit to mattermost-build/focalboard that referenced this pull request Jul 27, 2022
@mgdelacroix @mattermost-build
Move category creation and UUID update to a more performant data migr…
7436615 
…ation (mattermost-community#3437)

* Move category creation and UUID update to a more performant data migration

* Fix linter

* Fix linter

(cherry picked from commit 6616a16)
@mattermost-build mattermost-build mentioned this pull request Jul 27, 2022
Merged
@mattermod mattermod added CherryPick/Done Successfully cherry-picked to the quality or patch release tracked in the milestone and removed CherryPick/Approved Meant for the quality or patch release tracked in the milestone labels Jul 27, 2022
mgdelacroix added a commit that referenced this pull request Jul 27, 2022
@mattermost-build @mgdelacroix
Move category creation and UUID update to a more performant data migr…
97eaf80 
…ation (#3437) (#3439)

* Move category creation and UUID update to a more performant data migration

* Fix linter

* Fix linter

(cherry picked from commit 6616a16)

Co-authored-by: Miguel de la Cruz <miguel@mcrx.me>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sbishel sbishel sbishel approved these changes

@jespino jespino jespino approved these changes

@harshilsharma63 harshilsharma63 Awaiting requested review from harshilsharma63

Assignees
No one assigned
Labels
2: Dev Review Requires review by a core committer CherryPick/Done Successfully cherry-picked to the quality or patch release tracked in the milestone
Projects
None yet
Milestone
v7.2
Development

Successfully merging this pull request may close these issues.
4 participants
@mgdelacroix @mattermod @jespino @sbishel