Skip to content

DM sync unit will corrupt data when upstream data is latin1 charset and code point > 7F #7028

Closed
@lance6716

Description

What did you do?

https://github.com/pingcap/tiflow/pull/7027/files

insert data Müller to a latin1 column at sync unit

insert into t5 (id, name) values (1, 'Müller');

What did you expect to see?

no data corript

What did you see instead?

becomes M�ller

MySQL [(none)]> select * from gbk.t5
    -> ;
+----+---------+---------+
| id | name    | name2   |
+----+---------+---------+
|  0 | Müller  | Müller  |
|  1 | M�ller   | Müller  |
+----+---------+---------+
2 rows in set (0.003 sec)

MySQL [(none)]> show create table gbk.t5;
+-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table | Create Table                                                                                                                                                                                                                               |
+-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| t5    | CREATE TABLE `t5` (
  `id` int(11) NOT NULL,
  `name` varchar(20) DEFAULT NULL,
  `name2` varchar(20) DEFAULT 'Müller',
  PRIMARY KEY (`id`) /*T![clustered_index] CLUSTERED */
) ENGINE=InnoDB DEFAULT CHARSET=latin1 COLLATE=latin1_bin  |
+-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.001 sec)

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions