[MySQL] Automatic schema change handling

[Rentacookie]

Currently, when syncing from a MySQL database database schema changes are ignored. This is problematic when those changes affect tables that are in the defined sync rules. The work around for that has been to redeploy the sync rules, thereby triggering a full re-sync.

This change set adds functionality to automatically handle schema changes affecting tables in the sync rules by listening for and parsing DDL query binlog events.

The basic mechanism for this works as follows:

1. Listen for query binlog events.
2. Parse these events to see if they affect the schema of tables in the sync rules.
3. Pause the binlog listener and update the stored bucket data based on the schema change if required
4. Restart the binlog listener.

Parsing of the query SQL is accomplished by using the node-sql-parser package. This does introduce a limitation in that if the library can't parse the query, we cannot easily interpret the intended schema change either. If this is the case, some best effort pattern matching is done to check if the query is a schema change that affects one of the replicated tables and a warning is logged.

The following schema changes are detectable:

• Create, rename, drop Table
• Truncate Table
• Create / Drop unique indexes and primary keys
• Add, modify, drop, rename columns

With the exception of Create Table, all of the schema events are detected by parsing the DDL statements received in the binlog query events. For Create table, these changes are picked up when row events are received for the new table.

[Rentacookie]

This is more of code style change than anything else, we tend to rather create these encapsulating interfaces instead of having functions/constructors with a lot of parameters

[Rentacookie]

The original name here was not quite right and could be confusing. Reverted to the name primarily used throughout the code base

[Rentacookie]

This paves the way for some potential future cleanup, but also helps consolidating the naming of fields.

[Rentacookie]

This is a rather useful tag to include with our metrics.

[Rentacookie]

@rkistner I am not 100% sure if this is the correct place to fix it, but I found that without this, columns with null values would not be added to sqliterow and added to the storage.
There is a schema change test that adds a column that exposed that this might be a problem.

[Rentacookie]

Technically we can combine most of these column changes into one type, but keeping them seperate opens the door to some potential optimisations for handling the schema change.

[Rentacookie]

Whilst testing I discovered that trying to set up the heartbeat once the listener is already active does not work. The listener itself already has a lock on the connection then.

[Rentacookie]

This was required after decoupling the zongji stop/start from this binlog listener wrapper stop start.

Internally the binlog listener can stop and start Zongji when required, not affecting the externally viewed overal stop/start state of this wrapper.

[rkistner]

How closely does this parser follow the MySQL syntax? And what happens if it runs into syntax it doesn't support?

One example of a past issue:
taozhi8833998/node-sql-parser#2232

To what extent would it be feasible to use the tablemap events instead of or in addition to this to detect changes?

[rkistner]

Here's an example that crashes the process with a nasty error (thanks to LLMs for generating some nasty test cases):

CREATE TABLE sales (
    sale_date DATE,
    amount DECIMAL(10,2)
) PARTITION BY RANGE (YEAR(sale_date)) (
    PARTITION p2023 VALUES LESS THAN (2024),
    PARTITION p2024 VALUES LESS THAN (2025),
    PARTITION pmax VALUES LESS THAN MAXVALUE
);

The crash happens regardless of whether the table is referenced in sync rules or not, and you cannot recover by restarting - it will immediately crash at the same point again.

[rkistner]

Another example:

CREATE TABLE invisible_column_example (
  id INT,
  created_at DATE INVISIBLE
)

The point is not these specific examples though, just that it would be extremely difficult to cover all cases.

[Rentacookie]

I see what you mean. 😧

Unfortunately the tablemap events are really only useable for detecting some column modifications. Dropping, truncating and renaming of tables do not result in any tablemap events. I did initially try a joint approach with tablemap + query parsing. But in the end since you have to parse all DDL queries anyway it felt simpler to settle on a single source for schema changes.

So that left the query parsing as the only real source of those schema changes. But as you say, if we can't parse the DDL queries our hands are rather tied. In those cases we probably have to end up skipping that specific schema change 😬 Maybe we list that as a known issue and add a warning to the logs when it happens?
That hopefully doesn't happen too often though, considering that node-sql-parser is a rather prolificly used and maintained. 🤞

I am open to any other approaches though.

[rkistner]

What specific race conditions are covered here?

It feels like this has the possibility to introduce more race conditions, since the order is not preserved between schema changes (applied immediately here) and data modifications (queued).

Furthermore, zongi doesn't wait for these promises to complete AFAIK, so processing here can cause concurrent processing of events if multiple events come in shortly after each other, and uncaught rejections if the processing fails.

[rkistner]

Ok, reading more code I see it works like this:

1. Schema change detected (synchronously up to that point).
2. Stop the listener.
3. Wait until queue is empty.
4. Process the schema change.

So that would avoid the race conditions. I'm just wondering if it could be simpler to do this as part of the normal queue mechanism?

[Rentacookie]

I did try the normal queuing mechanism first, but the problem I ran into was that after handling the schema change we update the current binlogposition so that when the listener is restarted we process event from that point onwards. But there might already be a bunch of events in the queue, so if the listener is restarted with the binlog position post schema change. A bunch of duplicate events end up on the processing queue.

So that could be handled by processing all the events in the queue after the schema change, causing the binglog position to update and only then restarting the listener. But since the processing queue only has a single worker you run into a deadlock. As in you can't process any new tasks until the current schema change one has been completely finished and removed from the queue.

The schema change task can't technically be truly completed until the listener is restarted, but maybe I can work around that with a callback 🤔 I'll check it out since I much preferred the processing queue only approach.

[Rentacookie]

Good news I was able to slot the schema handling back into the processing queue, by using a callback to restart the listener once the queue had emptied.

[Rentacookie]

Adding this functionality fixed the issue where wild card tables could not be replicated

[Copilot]

Pull Request Overview

Adds automatic handling of schema changes (DDL events) in MySQL replication and standardizes the use of a name property instead of table across sync-rules and storage modules.

• Renamed SourceTableInterface.table to .name everywhere in sync-rules and service-core
• Introduced BinLogListener logic to parse DDL statements and emit schema change events
• Updated all storage and API layers (Postgres, MySQL, MongoDB) to use table.name and qualifiedName

Reviewed Changes

Copilot reviewed 41 out of 42 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
packages/sync-rules/src/…/SourceTableInterface.ts	Renamed table → name in interface
packages/service-core/src/storage/SourceTable.ts	Refactored SourceTable to take an options object
modules/module-mysql/src/replication/zongji/BinLogListener.ts	Added DDL parsing, schema change handling, restart logic
modules/module-postgres-storage/src/storage/batch/PostgresBucketBatch.ts	Updated mapping to use table.name
modules/module-mysql/test/src/BinLogStream.test.ts	Fixed missing await and test descriptions
modules/module-mysql/src/utils/parser-utils.ts	Introduced DDL‐matching helper
packages/service-core/src/metrics/open-telemetry/util.ts	Added service.version and JSON import of package.json

Files not reviewed (1)

• pnpm-lock.yaml: Language not supported

Comments suppressed due to low confidence (4)

packages/service-core/src/metrics/open-telemetry/util.ts:66

• Use the semantic key 'service.name' (per OpenTelemetry conventions) instead of 'service' for the service identifier.

        ['service']: 'PowerSync',

modules/module-mysql/test/src/BinLogStream.test.ts:16

• [nitpick] Typo in the test suite title: it should read 'BinLogStream tests' to match the class name.

describe('BigLogStream tests', () => {

packages/service-core/src/storage/SourceTable.ts:55

• [nitpick] The hasReplicaIdentity getter was removed; verify that no downstream code relies on it or reintroduce it if still used.

   */

modules/module-postgres-storage/src/storage/batch/PostgresBucketBatch.ts:467

• Confirm that the table object has a name property; if it still uses table.table, update the mapping to use table.table or rename the source field accordingly.

        name: table.name,

[Copilot]

Fix the typo in the error message: change 'No marching source query' to 'No matching source query'.

Suggested change

	errors: [{ error: `No marching source query found for table ${options.sourceTable.name}` }]
	errors: [{ error: `No matching source query found for table ${options.sourceTable.name}` }]