-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Move towards MySQL 8.0 as the default template generation #11153
Move towards MySQL 8.0 as the default template generation #11153
Conversation
This upgrades the remaining things to Ubuntu 20.04 and makes MySQL 8.0 the default we run tests against. We still have tests for MySQL 5.7 but those are now explicitly opted into. This should finish up the Ubuntu 20.04 upgrade and also makes things easier for the future when we need to upgrade again. Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
Review ChecklistHello reviewers! 👋 Please follow this checklist when reviewing this Pull Request. General
Bug fixes
Non-trivial changes
New/Existing features
Backward compatibility
|
…neration Signed-off-by: deepthi <deepthi@planetscale.com>
…for 8.0 Signed-off-by: deepthi <deepthi@planetscale.com>
…neration Signed-off-by: deepthi <deepthi@planetscale.com>
… 45 mins Signed-off-by: deepthi <deepthi@planetscale.com>
…nly need this for vtorc, but I've made the change to the template so all tests get it. Signed-off-by: deepthi <deepthi@planetscale.com>
…neration Signed-off-by: deepthi <deepthi@planetscale.com>
test/ci_workflow_gen.go
Outdated
@@ -121,15 +120,14 @@ var ( | |||
"vreplication_cellalias", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Line 117: do we still need a separate mysql80
test?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Everything else looks good to me!
@@ -93,7 +93,7 @@ func TestReadTopologyInstanceBufferable(t *testing.T) { | |||
assert.Contains(t, primaryInstance.InstanceAlias, "zone1") | |||
assert.NotEqual(t, 0, primaryInstance.ServerID) | |||
assert.Greater(t, len(primaryInstance.ServerUUID), 10) | |||
assert.Contains(t, primaryInstance.Version, "5.7") | |||
assert.Contains(t, primaryInstance.Version, "8.0") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can use assert.Regexp(t,
[58].[07].*, replicaInstance.Version)
instead. This will make this test work with both 5.7 and 8.0
Signed-off-by: deepthi <deepthi@planetscale.com>
…neration Signed-off-by: deepthi <deepthi@planetscale.com>
Signed-off-by: deepthi <deepthi@planetscale.com>
I've addressed the feedback, and except for the known issue with vtbackup_transform, tests are passing. |
…80-default-ci-template-generation Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>
Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>
Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
❤️
I was unable to backport this Pull Request to the following branches: |
…1153) * Move towards MySQL 8.0 as the default template generation This upgrades the remaining things to Ubuntu 20.04 and makes MySQL 8.0 the default we run tests against. We still have tests for MySQL 5.7 but those are now explicitly opted into. This should finish up the Ubuntu 20.04 upgrade and also makes things easier for the future when we need to upgrade again. Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> * CI: rename shard vtorc_8.0 to vtorc_5.7, change expected test output for 8.0 Signed-off-by: deepthi <deepthi@planetscale.com> * CI: increase timeout for 8.0 tests on the actual test step from 30 to 45 mins Signed-off-by: deepthi <deepthi@planetscale.com> * CI: increase timeout to 45 minutes for mysql57 tests too. We really only need this for vtorc, but I've made the change to the template so all tests get it. Signed-off-by: deepthi <deepthi@planetscale.com> * CI: fix vtorc test to work with both 5.7 and 8.0 Signed-off-by: deepthi <deepthi@planetscale.com> * CI: missed docker flag in mysql57 template, one more fix to vtorc test Signed-off-by: deepthi <deepthi@planetscale.com> * removing spaces from pb file Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * removing spaces in pb file part 2 Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> Signed-off-by: deepthi <deepthi@planetscale.com> Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> Co-authored-by: deepthi <deepthi@planetscale.com> Co-authored-by: Rameez Sajwani <rameezwazirali@hotmail.com> Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>
…1153) * Move towards MySQL 8.0 as the default template generation This upgrades the remaining things to Ubuntu 20.04 and makes MySQL 8.0 the default we run tests against. We still have tests for MySQL 5.7 but those are now explicitly opted into. This should finish up the Ubuntu 20.04 upgrade and also makes things easier for the future when we need to upgrade again. Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> * CI: rename shard vtorc_8.0 to vtorc_5.7, change expected test output for 8.0 Signed-off-by: deepthi <deepthi@planetscale.com> * CI: increase timeout for 8.0 tests on the actual test step from 30 to 45 mins Signed-off-by: deepthi <deepthi@planetscale.com> * CI: increase timeout to 45 minutes for mysql57 tests too. We really only need this for vtorc, but I've made the change to the template so all tests get it. Signed-off-by: deepthi <deepthi@planetscale.com> * CI: fix vtorc test to work with both 5.7 and 8.0 Signed-off-by: deepthi <deepthi@planetscale.com> * CI: missed docker flag in mysql57 template, one more fix to vtorc test Signed-off-by: deepthi <deepthi@planetscale.com> * removing spaces from pb file Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * removing spaces in pb file part 2 Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> Signed-off-by: deepthi <deepthi@planetscale.com> Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> Co-authored-by: deepthi <deepthi@planetscale.com> Co-authored-by: Rameez Sajwani <rameezwazirali@hotmail.com>
…1153) * Move towards MySQL 8.0 as the default template generation This upgrades the remaining things to Ubuntu 20.04 and makes MySQL 8.0 the default we run tests against. We still have tests for MySQL 5.7 but those are now explicitly opted into. This should finish up the Ubuntu 20.04 upgrade and also makes things easier for the future when we need to upgrade again. Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> * CI: rename shard vtorc_8.0 to vtorc_5.7, change expected test output for 8.0 Signed-off-by: deepthi <deepthi@planetscale.com> * CI: increase timeout for 8.0 tests on the actual test step from 30 to 45 mins Signed-off-by: deepthi <deepthi@planetscale.com> * CI: increase timeout to 45 minutes for mysql57 tests too. We really only need this for vtorc, but I've made the change to the template so all tests get it. Signed-off-by: deepthi <deepthi@planetscale.com> * CI: fix vtorc test to work with both 5.7 and 8.0 Signed-off-by: deepthi <deepthi@planetscale.com> * CI: missed docker flag in mysql57 template, one more fix to vtorc test Signed-off-by: deepthi <deepthi@planetscale.com> * removing spaces from pb file Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * removing spaces in pb file part 2 Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> Signed-off-by: deepthi <deepthi@planetscale.com> Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> Co-authored-by: deepthi <deepthi@planetscale.com> Co-authored-by: Rameez Sajwani <rameezwazirali@hotmail.com>
…11619) * Move towards MySQL 8.0 as the default template generation (#11153) * Move towards MySQL 8.0 as the default template generation This upgrades the remaining things to Ubuntu 20.04 and makes MySQL 8.0 the default we run tests against. We still have tests for MySQL 5.7 but those are now explicitly opted into. This should finish up the Ubuntu 20.04 upgrade and also makes things easier for the future when we need to upgrade again. Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> * CI: rename shard vtorc_8.0 to vtorc_5.7, change expected test output for 8.0 Signed-off-by: deepthi <deepthi@planetscale.com> * CI: increase timeout for 8.0 tests on the actual test step from 30 to 45 mins Signed-off-by: deepthi <deepthi@planetscale.com> * CI: increase timeout to 45 minutes for mysql57 tests too. We really only need this for vtorc, but I've made the change to the template so all tests get it. Signed-off-by: deepthi <deepthi@planetscale.com> * CI: fix vtorc test to work with both 5.7 and 8.0 Signed-off-by: deepthi <deepthi@planetscale.com> * CI: missed docker flag in mysql57 template, one more fix to vtorc test Signed-off-by: deepthi <deepthi@planetscale.com> * removing spaces from pb file Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * removing spaces in pb file part 2 Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> Signed-off-by: deepthi <deepthi@planetscale.com> Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> Co-authored-by: deepthi <deepthi@planetscale.com> Co-authored-by: Rameez Sajwani <rameezwazirali@hotmail.com> * fixing template files to default to mysql80 Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * changing update statement to alter Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * Avoid deadlocks related to 0 receiver behavior (#10132) * Prevent deadlocks related to 0 receiver behavior Signed-off-by: Matt Lord <mattalord@gmail.com> * Update test tables to use poller_idx Signed-off-by: Matt Lord <mattalord@gmail.com> * Minor changes after mutex usage review in message manager + cache Signed-off-by: Matt Lord <mattalord@gmail.com> * Use atomics for receiver count and messages pending Signed-off-by: Matt Lord <mattalord@gmail.com> * Don't take exclusive lock when in fast path Signed-off-by: Matt Lord <mattalord@gmail.com> * Update tests to use the new recommended message table structure See: vitessio/website#1015 Signed-off-by: Matt Lord <mattalord@gmail.com> * Correct tests Signed-off-by: Matt Lord <mattalord@gmail.com> * Update e2e test to use new recommended table structure Signed-off-by: Matt Lord <mattalord@gmail.com> * Fix TestMessageStreamingPlan test Signed-off-by: Matt Lord <mattalord@gmail.com> * Fix godriver/TestStreamMessaging test Signed-off-by: Matt Lord <mattalord@gmail.com> * Split streamMu into streamProcessingMu and lastPollPositionMu Signed-off-by: Matt Lord <mattalord@gmail.com> * Poller cannot take main lock w/o having X stream processing lock Signed-off-by: Matt Lord <mattalord@gmail.com> * Improve the comments a bit Signed-off-by: Matt Lord <mattalord@gmail.com> * Hold the main mutex during Add This is for safe concurrency with the last receiver unsubscribing Signed-off-by: Matt Lord <mattalord@gmail.com> * Changes after pair reviewing with Sugu Signed-off-by: Matt Lord <mattalord@gmail.com> * Use my GitHub handle for the self reference Signed-off-by: Matt Lord <mattalord@gmail.com> Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * remove unwanted sleep Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * Fix failures Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * Fix backup tests Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * fix vtgate_gen4 error Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * upgrading to mysql 8.0 using vinalla mysql Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * pin to specific version 8.0.25 Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * Try to pin mysql version to 8.0.25 using tar file Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * Fix bug in last commit for 8.0.25 Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * fixing template files to use mysql8.0.25 Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * fixing community version Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * removing all mysql version before installation Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * moving cluster 11 to self host Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * trying different combination since mysql is not getting installed correctly Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * use tar instead of deb file Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * remove mysql stop statement Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * setting vt_mysql_root Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * moving export to right place Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * change template to accomodate tar file download Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * fixing mysql80 cluster Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * Adding mysql to the path env variable Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * fixing shardedpitr_tls test Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * fix upgrade-downgrade test Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * adjust epected AUTO_INCREMENT value Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * removed 'expected_table_structure' files because there are different outputs in mysql57 and in mysql80 Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * adding mysql version in workflow logs Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * move to utuntu 18 for upgrade-downgrade test Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> Signed-off-by: deepthi <deepthi@planetscale.com> Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> Signed-off-by: Matt Lord <mattalord@gmail.com> Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> Co-authored-by: Dirkjan Bussink <d.bussink@gmail.com> Co-authored-by: deepthi <deepthi@planetscale.com> Co-authored-by: Matt Lord <mattalord@gmail.com> Co-authored-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
* Move towards MySQL 8.0 as the default template generation (#11153) * Move towards MySQL 8.0 as the default template generation This upgrades the remaining things to Ubuntu 20.04 and makes MySQL 8.0 the default we run tests against. We still have tests for MySQL 5.7 but those are now explicitly opted into. This should finish up the Ubuntu 20.04 upgrade and also makes things easier for the future when we need to upgrade again. Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> * CI: rename shard vtorc_8.0 to vtorc_5.7, change expected test output for 8.0 Signed-off-by: deepthi <deepthi@planetscale.com> * CI: increase timeout for 8.0 tests on the actual test step from 30 to 45 mins Signed-off-by: deepthi <deepthi@planetscale.com> * CI: increase timeout to 45 minutes for mysql57 tests too. We really only need this for vtorc, but I've made the change to the template so all tests get it. Signed-off-by: deepthi <deepthi@planetscale.com> * CI: fix vtorc test to work with both 5.7 and 8.0 Signed-off-by: deepthi <deepthi@planetscale.com> * CI: missed docker flag in mysql57 template, one more fix to vtorc test Signed-off-by: deepthi <deepthi@planetscale.com> * removing spaces from pb file Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * removing spaces in pb file part 2 Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> Signed-off-by: deepthi <deepthi@planetscale.com> Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> Co-authored-by: deepthi <deepthi@planetscale.com> Co-authored-by: Rameez Sajwani <rameezwazirali@hotmail.com> * changing update statement to alter Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * Adding mysql 8.0.25 Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * fix junit Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * fixing failures Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * fixing more unit tests Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * fix selfhost test Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * Fixing selfhosted workflows Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * fixing vreplication_v2 Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * removing cluster 12 and 18 from selfhosted Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * manual fixing some workflows Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * making vreplication_v2 exactly like upstream Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * Moving all the workflow to latest mysql version (vanilla) Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * remove port restrictions Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * tmp fix for vreplication_v2 Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * making vreplication_v2 to previous version Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * correcting typo Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * mvoing vreplication_v2 to selfhost Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * change docker image to mysql80 Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * removing limited source from vrepl Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * revert back last commit changes Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * test: fix aggregation test Signed-off-by: Harshit Gangal <harshit@planetscale.com> * removing ptrtls Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * remove pitrTls tests Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * changing source/destination to mysql8.0 for vreplication Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * remove unwanted workflows Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * adding timeout to stress test Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * feeback through code review Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> Signed-off-by: deepthi <deepthi@planetscale.com> Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> Signed-off-by: Harshit Gangal <harshit@planetscale.com> Co-authored-by: Dirkjan Bussink <d.bussink@gmail.com> Co-authored-by: deepthi <deepthi@planetscale.com> Co-authored-by: Harshit Gangal <harshit@planetscale.com>
This upgrades the remaining things to Ubuntu 20.04 and makes MySQL 8.0 the default we run tests against. We still have tests for MySQL 5.7 but those are now explicitly opted into.
This should finish up the Ubuntu 20.04 upgrade and also makes things easier for the future when we need to upgrade again.
It is also marked as back port since we want to keep this updated on back ports as well or otherwise we're be blocked in the somewhat near future when 18.04 is deprecated. We probably want to look at changing the default maybe back there to 5.7 (to minimize other test changes that happened in previous PRs like #11118) but that's a separate step to look at during back porting.
Related Issue(s)
Part of the upgrade already started with #11026 but keeping things sensible in the CI generation scripts is much easier with this upgrade we think.
Checklist