modify column on partitioned table does not give warning about truncated rows.

[mjonss]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

drop table if exists t;
set sql_mode = default;
create table t (a varchar(255)) partition by range columns (a) (partition p1 values less than ("0"), partition p2 values less than ("zzzzzz"));
insert into t values ("123456");
alter table t modify a varchar(5);
set sql_mode = '';
alter table t modify a varchar(5);
show warnings;

2. What did you expect to see? (Required)

Note the change in p1 partition definition, leading to the truncated value is in first partition!

tidb> create table t (a varchar(255)) partition by range columns (a) (partition p1 values less than ("2"), partition p2 values less than ("zzzzzz"));
--------------
create table t (a varchar(255)) partition by range columns (a) (partition p1 values less than ("2"), partition p2 values less than ("zzzzzz"))
--------------

Query OK, 0 rows affected (0.16 sec)

tidb> insert into t values ("123456");
--------------
insert into t values ("123456")
--------------

Query OK, 1 row affected (0.02 sec)

tidb> alter table t modify a varchar(5);
--------------
alter table t modify a varchar(5)
--------------

ERROR 1265 (01000): Data truncated for column 'a', value is '123456'
tidb> set sql_mode = '';
--------------
set sql_mode = ''
--------------

Query OK, 0 rows affected (0.00 sec)

tidb> alter table t modify a varchar(5);
--------------
alter table t modify a varchar(5)
--------------

Query OK, 0 rows affected, 1 warning (2.96 sec)

tidb> show warnings;
--------------
show warnings
--------------

+---------+------+--------------------------------------------------+
| Level   | Code | Message                                          |
+---------+------+--------------------------------------------------+
| Warning | 1265 | Data truncated for column 'a', value is '123456' |
+---------+------+--------------------------------------------------+
1 row in set (0.00 sec)

3. What did you see instead (Required)

tidb> create table t (a varchar(255)) partition by range columns (a) (partition p1 values less than ("0"), partition p2 values less than ("zzzzzz"));
--------------
create table t (a varchar(255)) partition by range columns (a) (partition p1 values less than ("0"), partition p2 values less than ("zzzzzz"))
--------------

Query OK, 0 rows affected (0.12 sec)

tidb> insert into t values ("123456");
--------------
insert into t values ("123456")
--------------

Query OK, 1 row affected (0.05 sec)

tidb> alter table t modify a varchar(5);
--------------
alter table t modify a varchar(5)
--------------

ERROR 1265 (01000): Data truncated for column 'a', value is '123456'
tidb> set sql_mode = '';
--------------
set sql_mode = ''
--------------

Query OK, 0 rows affected (0.00 sec)

tidb> alter table t modify a varchar(5);
--------------
alter table t modify a varchar(5)
--------------

Query OK, 0 rows affected (3.06 sec)

tidb> show warnings;
--------------
show warnings
--------------

Empty set (0.00 sec)

4. What is your TiDB version? (Required)

tidb_version(): Release Version: v6.4.0-alpha
Edition: Community
Git Commit Hash: 753e3da
Git Branch: heads/refs/tags/v6.4.0-alpha
UTC Build Time: 2022-10-26 14:24:52
GoVersion: go1.19
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: tikv

[mjonss]

I don't think this is a new bug or related to table partitioning.

OK, so here is what I think happens:

1. in updateColumnWorker.BackfillDataInTxn it updates rows in-place (adds + deletes in batched transactions) and collects the warnings into the taskCtx (for further propagation).
2. something happens (not sure why this is triggered in this case?) resulting in the updateDDLJob2Table cannot commit the update to mysql.tidb_ddl_reorg containing the warnings, so they will get lost.
3. new run of updateColumnWorker.BackfillDataInTxn, which will not find the rows causing errors or warnings, since they have already been changed (truncated) and committed in 1)

To be sure that the errors/warnings are stored, they need to be included in the commit together with the row/index updates in 1)

[mjonss]

In 2) this is the root cause:
a) In HandleDDLJobTable a transaction is started, then it calls runDDLJob
b) runDDLJob may start other transactions, possibly in new goroutines, which also will commit changes to tidb_ddl_reorg table (at least for keeping which startkey and possibly which partition) for the jobID
c) When returning to HandleDDLJobTable it will call updateDDLJob which will also write to the mysql.tidb_ddl_reorg table updating the status for the same jobID. This conflicts with the commit in b), resulting in a non-retryable commit failure, and the job will restart soon, but the warnings (and possibly other status is lost).

Similar behaviour can also be observed in master, running the ddl/TestPartitionAddIndex test (but without the lost warnings, just that the transaction fails).