Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ddl: fix alter table exchange partition does not work if table has tiflash replica #17940

Merged
merged 97 commits into from
Jun 28, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
97 commits
Select commit Hold shift + click to select a range
70f6354
exchange partition init push
zhaox1n May 4, 2020
57af241
exchange partition: check table is compatiable
zhaox1n May 4, 2020
af97469
Merge pull request #1 from pingcap/master
zhaox1n May 4, 2020
adef48e
add checke in ddl worker
zhaox1n May 5, 2020
1a7f599
validate check
zhaox1n May 6, 2020
11809d0
Basic functions have been completed
zhaox1n May 6, 2020
c8bc050
autoID added
zhaox1n May 7, 2020
5a32fe4
test add
zhaox1n May 8, 2020
35941aa
hash exchange partition add
zhaox1n May 9, 2020
7468924
format
zhaox1n May 9, 2020
64fd709
test add
zhaox1n May 9, 2020
cb88c18
Merge pull request #2 from pingcap/master
zhaox1n May 9, 2020
71cf840
test add
zhaox1n May 10, 2020
a902cdc
auto id fix
zhaox1n May 10, 2020
a228468
add cancel job test
zhaox1n May 10, 2020
9ff0790
add exchange partition compatiable test
zhaox1n May 11, 2020
dda9d75
add test of partition
zhaox1n May 12, 2020
32c6285
add more test and set go mod
zhaox1n May 13, 2020
899033f
deal with conflict
zhaox1n May 13, 2020
f4f5b38
fix the conflict
zhaox1n May 13, 2020
a38b5e0
deal with conflict
zhaox1n May 14, 2020
3cec6f2
add more test of table compatible
zhaox1n May 14, 2020
53f4f73
fix comment
zhaox1n May 14, 2020
101c673
rename function
zhaox1n May 14, 2020
a1ca360
fix wrong english spell
zhaox1n May 14, 2020
40db78a
Merge remote-tracking branch 'upstream/master'
zhaox1n May 16, 2020
c559f12
add test for expression index
zhaox1n May 16, 2020
f9fdd0c
add check in ddl worker
zhaox1n May 17, 2020
c979e69
rename word
zhaox1n May 18, 2020
c096c8c
add global config of expression index
zhaox1n May 18, 2020
4130a9a
deal conflict
zhaox1n May 18, 2020
0ffd1ab
deal conflict
zhaox1n May 21, 2020
488cb5f
add auto_random test
zhaox1n May 21, 2020
b5cfdd3
comment fix
zhaox1n May 26, 2020
fac04c1
comment fix
zhaox1n May 26, 2020
be92bb3
more test added
zhaox1n May 26, 2020
3f38e2b
Merge remote-tracking branch 'upstream/master'
zhaox1n May 26, 2020
d9bc6fa
add failed point
zhaox1n May 26, 2020
ea173bf
remove variable
zhaox1n May 26, 2020
57aeb95
Update ddl/column.go
zhaox1n May 27, 2020
5a29bdd
Update ddl/ddl_api.go
zhaox1n May 27, 2020
aa8da8e
comment fix
zhaox1n May 27, 2020
ebce341
add test
zhaox1n May 27, 2020
65f4c64
add test
zhaox1n May 27, 2020
5a46b18
exchange auto_random
zhaox1n May 27, 2020
ec51f5e
fix range columns partition
zhaox1n May 28, 2020
a93fe24
Merge remote-tracking branch 'upstream/master'
zhaox1n May 28, 2020
95b2554
fix comment
zhaox1n May 29, 2020
134c55d
fix comment
zhaox1n May 29, 2020
d724823
fix comment
zhaox1n May 29, 2020
894b7e5
fix bug
zhaox1n May 30, 2020
40729a1
fix comment
zhaox1n Jun 1, 2020
6ea633d
fix comment
zhaox1n Jun 1, 2020
d9b98c0
fix comment
zhaox1n Jun 1, 2020
b194f11
Merge remote-tracking branch 'upstream/master' into exchange_partitio…
zhaox1n Jun 1, 2020
7807380
fix comment
zhaox1n Jun 1, 2020
4769d3b
bug fix
zhaox1n Jun 2, 2020
325d596
address comment
zhaox1n Jun 2, 2020
33e0c36
Merge pull request #3 from zhaox1n/exchange_partition_test
zhaox1n Jun 2, 2020
6eeb3fd
Merge branch 'master' into master
crazycs520 Jun 2, 2020
b46a6bb
Merge branch 'master' into master
tiancaiamao Jun 3, 2020
85d71c0
address comment
zhaox1n Jun 4, 2020
a4d0161
Merge branch 'master' of https://github.com/zhaox1n/tidb
zhaox1n Jun 4, 2020
6c1d63a
address comment
zhaox1n Jun 6, 2020
b5b6824
bug fix
zhaox1n Jun 6, 2020
999b0d9
bug fix
zhaox1n Jun 6, 2020
2cf5d03
sql fix
zhaox1n Jun 6, 2020
52676f8
add test of auto_random
zhaox1n Jun 6, 2020
6bc1a4a
expand the implementation of ApplyDiff
zhaox1n Jun 7, 2020
9cd7caa
fix typo
zhaox1n Jun 7, 2020
771e4dd
Update ddl/partition.go
zhaox1n Jun 8, 2020
f2ab275
address conflict
zhaox1n Jun 8, 2020
dc8abd3
Merge branch 'master' into master
crazycs520 Jun 9, 2020
44f6018
address comment
zhaox1n Jun 9, 2020
3ec7b7a
merge master
zhaox1n Jun 9, 2020
f3fc2e7
address comment
zhaox1n Jun 10, 2020
273e911
fix bug of exchanging partition on tiflash replica
zhaox1n Jun 10, 2020
ffe48c0
merge master
zhaox1n Jun 11, 2020
bc77a30
merge master
zhaox1n Jun 11, 2020
db68d2a
delete duplicate code
zhaox1n Jun 11, 2020
437aab9
Merge branch 'master' into master
zhaox1n Jun 15, 2020
f7bd233
merge origin master
zhaox1n Jun 20, 2020
62b7e27
fix conclict
zhaox1n Jun 20, 2020
0acdba0
reset tiflash statu
zhaox1n Jun 20, 2020
134109d
Update ddl/ddl_api.go
zhaox1n Jun 22, 2020
aa789e8
Update ddl/ddl_api.go
zhaox1n Jun 22, 2020
72438b2
address comment
zhaox1n Jun 22, 2020
6a31a34
Merge branch 'master' into master
zhaox1n Jun 23, 2020
46115cb
exchange partition init push
zhaox1n May 4, 2020
ee85e37
fix bug of exchanging partition on tiflash replica
zhaox1n Jun 10, 2020
0858658
reset tiflash statu
zhaox1n Jun 20, 2020
2b74ae4
rebase
zhaox1n Jun 23, 2020
30b726a
reset mod
zhaox1n Jun 23, 2020
245cd92
address comment
zhaox1n Jun 26, 2020
ce74f1b
Merge branch 'master' into master
zhaox1n Jun 27, 2020
4dafc9b
address comment
zhaox1n Jun 28, 2020
f838184
Merge branch 'master' into master
zhaox1n Jun 28, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
reset tiflash statu
  • Loading branch information
zhaox1n committed Jun 20, 2020
commit 0acdba033ed81db6d058f7e45bfa3cef2c306d2d
56 changes: 55 additions & 1 deletion ddl/db_partition_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -940,15 +940,70 @@ func (s *testIntegrationSuite4) TestAlterTableExchangePartition(c *C) {
tk.MustExec("create table e16 (a int)")
tk.MustExec("alter table e15 set tiflash replica 1;")
tk.MustExec("alter table e16 set tiflash replica 2;")

e15 := testGetTableByName(c, s.ctx, "test", "e15")
partition := e15.Meta().Partition

err := domain.GetDomain(tk.Se).DDL().UpdateTableReplicaInfo(tk.Se, partition.Definitions[0].ID, true)
c.Assert(err, IsNil)

e16 := testGetTableByName(c, s.ctx, "test", "e16")
err = domain.GetDomain(tk.Se).DDL().UpdateTableReplicaInfo(tk.Se, e16.Meta().ID, true)
c.Assert(err, IsNil)

tk.MustGetErrCode("alter table e15 exchange partition p0 with table e16", tmysql.ErrTablesDifferentMetadata)
tk.MustExec("drop table e15, e16")

tk.MustExec("create table e15 (a int) partition by hash(a) partitions 1;")
tk.MustExec("create table e16 (a int)")
tk.MustExec("alter table e15 set tiflash replica 1;")
tk.MustExec("alter table e16 set tiflash replica 1;")

e15 = testGetTableByName(c, s.ctx, "test", "e15")
partition = e15.Meta().Partition

err = domain.GetDomain(tk.Se).DDL().UpdateTableReplicaInfo(tk.Se, partition.Definitions[0].ID, true)
c.Assert(err, IsNil)

e16 = testGetTableByName(c, s.ctx, "test", "e16")
err = domain.GetDomain(tk.Se).DDL().UpdateTableReplicaInfo(tk.Se, e16.Meta().ID, true)
c.Assert(err, IsNil)

tk.MustExec("alter table e15 exchange partition p0 with table e16")

e15 = testGetTableByName(c, s.ctx, "test", "e15")

partition = e15.Meta().Partition

c.Assert(e15.Meta().TiFlashReplica, NotNil)
c.Assert(e15.Meta().TiFlashReplica.Available, IsFalse)
c.Assert(e15.Meta().TiFlashReplica.AvailablePartitionIDs, DeepEquals, []int64{partition.Definitions[0].ID})

e16 = testGetTableByName(c, s.ctx, "test", "e16")
c.Assert(e16.Meta().TiFlashReplica, NotNil)
c.Assert(e16.Meta().TiFlashReplica.Available, IsFalse)

tk.MustExec("drop table e15, e16")
tk.MustExec("create table e15 (a int) partition by hash(a) partitions 1;")
tk.MustExec("create table e16 (a int)")
tk.MustExec("alter table e16 set tiflash replica 1;")

tk.MustExec("alter table e15 set tiflash replica 1 location labels 'a', 'b';")

tk.MustGetErrCode("alter table e15 exchange partition p0 with table e16", tmysql.ErrTablesDifferentMetadata)

tk.MustExec("alter table e16 set tiflash replica 1 location labels 'a', 'b';")

e15 = testGetTableByName(c, s.ctx, "test", "e15")
partition = e15.Meta().Partition

err = domain.GetDomain(tk.Se).DDL().UpdateTableReplicaInfo(tk.Se, partition.Definitions[0].ID, true)
c.Assert(err, IsNil)

e16 = testGetTableByName(c, s.ctx, "test", "e16")
err = domain.GetDomain(tk.Se).DDL().UpdateTableReplicaInfo(tk.Se, e16.Meta().ID, true)
c.Assert(err, IsNil)

tk.MustExec("alter table e15 exchange partition p0 with table e16")
}

Expand Down Expand Up @@ -1149,7 +1204,6 @@ func (s *testIntegrationSuite4) TestExchangePartitionTableCompatiable(c *C) {
tk.MustExec(t.exchangeSQL)
}
}

}

func (s *testIntegrationSuite7) TestExchangePartitionExpressIndex(c *C) {
Expand Down
13 changes: 13 additions & 0 deletions ddl/partition.go
Original file line number Diff line number Diff line change
Expand Up @@ -812,6 +812,19 @@ func (w *worker) onExchangeTablePartition(d *ddlCtx, t *meta.Meta, job *model.Jo
// exchange table meta id
partDef.ID = nt.ID

// Clear the tiflash replica available status.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should remove this comment since the code no longer clear the tiflash replica available status

if pt.TiFlashReplica != nil {
pt.TiFlashReplica.Available = false
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why set Available to false? I think if exchange partition is allowed, the TiFlash replica status of both table are always the same(both available or both not available), it is not necessary to reset the TiFlash replica status to false if they are already available.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more question: is it possible that the available flag is changed between checkTiFlashReplicaCompatible and onExchangeTablePartition , if yes, then we need handle it more carefully.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get that It seem that tiflash now doesnot support exchange partition in slack. So I just reset false like truncate partition. I would delete the code about changing tiflash status.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TiFlash does not support exchange partition when #17149 is merged, and it has supported now, so you can safely delete related code, but take care of my last question, if available flag is changed, we need to handle the status carefully for both partition table and non-partition table.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hava a question about how to change a available status when a exchange partition happening ? It seems that only one worker handle ddl events.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hava a question about how to change a available status when a exchange partition happening ? It seems that only one worker handle ddl events.

I'm not very familiar with TiDB, if available status can not be changed during exchange partition, then just ignore my question.

nt.TiFlashReplica.Available = false
// Set partition replica become unavailable.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

Copy link
Contributor Author

@zhaox1n zhaox1n Jun 28, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done PTAL

for i, id := range pt.TiFlashReplica.AvailablePartitionIDs {
if id == tempID {
pt.TiFlashReplica.AvailablePartitionIDs[i] = partDef.ID
break
}
}
}

err = t.UpdateTable(ptSchemaID, pt)
if err != nil {
job.State = model.JobStateCancelled
Expand Down