Skip to content

Decode Exception due to Inaccurate Length from self.query_length Leading to Byte Sequence Truncation #601

Closed
@zj5220924

Description

@zj5220924

Version

Please specify the versions you are using. Exact version numbers are preferred.

  • Pymyrepl (1.1.0):
  • OS (Centos 7):
  • Pymyrepl (1.0.6):
  • Database and version (Remove unnecessary options):
    • MySQL: 8.0.32

System Variables

List relevant system variables using the query SHOW VARIABLES LIKE '<variable_name_here>';

  • log_bin: YES
  • binlog_format: ROW
  • binlog_row_image: FULL
  • enforce_gtid_consistency: True
  • gtid_mode: YES

If you are using MySQL 8.0.14 or later, please also provide the following variables:

  • binlog_row_metadata: FULL
  • binlog_row_value_options: Null

error massage:
Traceback (most recent call last):
File "/ROOT/tmp/my2sql/my2sql.py", line 144, in
binlog2sql.process_binlog()
File "/ROOT/tmp/my2sql/my2sql.py", line 72, in process_binlog
for binlog_event in stream:
File "/usr/local/python3.11/lib/python3.11/site-packages/pymysqlreplication/binlogstream.py", line 617, in fetchone
binlog_event = BinLogPacketWrapper(
^^^^^^^^^^^^^^^^^^^^
File "/usr/local/python3.11/lib/python3.11/site-packages/pymysqlreplication/packet.py", line 115, in init
self.event = event_class(
^^^^^^^^^^^^
File "/usr/local/python3.11/lib/python3.11/site-packages/pymysqlreplication/event.py", line 885, in init
self.query = self.packet.read(self.query_length).decode("utf-8")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 205-206: unexpected end of data

cat /....../pymysqlreplication/event.py
......
def init(self, from_packet, event_size, table_map, ctl_connection, **kwargs):
super(RowsQueryLogEvent, self).init(
from_packet, event_size, table_map, ctl_connection, **kwargs
)
self.query_length = self.packet.read_uint8()
self.query = self.packet.read(self.query_length).decode("utf-8")

print(self.query_length)
print(self.query)
......

After testing, it was found that the length obtained from self.query_length is inaccurate, causing the byte sequence read by self.packet.read to be truncated, leading to decoding exceptions.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions