fix(mysql-loader): refresh REPEATABLE READ snapshot before each query - #381
Merged
Chenglong Wang (Chenglong-MS) merged 2 commits intoJul 17, 2026
Conversation
The MySQLDataLoader keeps a long-lived PyMySQL connection with the default isolation level (REPEATABLE READ) and autocommit disabled. The first SELECT in such a session establishes a read view, and subsequent SELECTs against the same connection keep returning rows from that snapshot even after other sessions have committed new rows. As a result, /api/connectors/refresh-data would not see newly inserted rows unless the Data Formulator process was restarted, which broke the 'refresh connector table' UI affordance. Force a commit before every cursor.execute so that any open implicit transaction is rolled forward and the next SELECT establishes a new read view, picking up concurrent commits.
Copilot started reviewing on behalf of
Chenglong Wang (Chenglong-MS)
July 17, 2026 06:38
View session
Chenglong Wang (Chenglong-MS)
approved these changes
Jul 17, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes stale reads from MySQLDataLoader caused by MySQL鈥檚 default REPEATABLE READ behavior on a long-lived connection by rolling forward the implicit transaction before executing each query, so subsequent reads observe rows committed by other sessions (e.g., for /api/connectors/refresh-data).
Changes:
- Commit the MySQL connection before each
cursor.execute()in_read_sqlto force a fresh snapshot for subsequentSELECTs.
Comment on lines
+135
to
+138
| try: | ||
| conn.commit() | ||
| except Exception: | ||
| pass |
Comment on lines
+132
to
+136
| # Force a fresh REPEATABLE READ snapshot on every read so rows | ||
| # committed by other sessions after this loader was first | ||
| # instantiated are visible without restarting the process. | ||
| try: | ||
| conn.commit() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
MySQLDataLoaderkeeps a long-lived PyMySQL connection with the default isolation level (REPEATABLE READ) andautocommit=False. The firstSELECTagainst such a connection establishes a read view, and subsequentSELECTs keep returning rows from that snapshot even after other sessions have committed new rows.That made
/api/connectors/refresh-databehave as if a connector table never changed unless the Data Formulator process was restarted, breaking the "refresh table" affordance in the UI.Fix
Call
conn.commit()before everycursor.execute()so any open implicit transaction is rolled forward and the nextSELECTestablishes a fresh read view, picking up rows committed by other sessions in the meantime.Verification
world.citywith 4079 rows).ID=88888).POST /api/connectors/refresh-data.row_countwent from4079to4080anddata_changed=true. The re-writtenworld_city.parquetnow containsID=88888.Risk
Low.
conn.commit()on a connection that has not started a transaction is a no-op, and no transaction is in flight whencur.executeis about to start a new one. The change does not affect any other loader (only the MySQL path).