Skip to content

Comments

Revert "Allow partial sync after loading AOF with preamble (#2366)"#2925

Merged
ranshid merged 1 commit intovalkey-io:unstablefrom
zuiderkwast:revert-psync-after-load-aof
Dec 11, 2025
Merged

Revert "Allow partial sync after loading AOF with preamble (#2366)"#2925
ranshid merged 1 commit intovalkey-io:unstablefrom
zuiderkwast:revert-psync-after-load-aof

Conversation

@zuiderkwast
Copy link
Contributor

This reverts commit 2da21d9.

The implementation in #2366 made it possible to perform a partial resynchronization after loading an AOF file, by storing the replication offset in the AOF preamble and counting the bytes in the AOF command stream in the same way as we count byte offset in a replication stream.

However, this approach isn't safe because some commands are replicated but not stored in the AOF file. This includes the commands REPLCONF, PING, PUBLISH and module-implemented commands where a module can control to propagate a command to the replication stream only, to AOF only or to both. This oversight led to data inconsistency, where the wrong replication offset is used for partial resynchronization as explained in issue #2904.

The revert caused small merge conflicts with 3c3a196 which are solved.

Fixes #2904.

…#2366)"

This reverts commit 2da21d9.

Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
@zuiderkwast
Copy link
Contributor Author

@arthurkiller

@codecov
Copy link

codecov bot commented Dec 10, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 72.47%. Comparing base (44dc581) to head (60b6411).
⚠️ Report is 1 commits behind head on unstable.

Additional details and impacted files
@@             Coverage Diff              @@
##           unstable    #2925      +/-   ##
============================================
+ Coverage     72.44%   72.47%   +0.02%     
============================================
  Files           129      129              
  Lines         70538    70524      -14     
============================================
+ Hits          51104    51112       +8     
+ Misses        19434    19412      -22     
Files with missing lines Coverage Δ
src/aof.c 81.14% <100.00%> (-0.05%) ⬇️
src/rdb.c 77.14% <ø> (-0.20%) ⬇️
src/rdb.h 100.00% <ø> (ø)
src/server.c 88.55% <100.00%> (+0.18%) ⬆️

... and 14 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Contributor

@murphyjacob4 murphyjacob4 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revert LGTM. Caught up on the context and revert sounds like the correct approach until we revisit PSYNC from AOF design

Copy link
Member

@ranshid ranshid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ranshid ranshid merged commit 5940dbf into valkey-io:unstable Dec 11, 2025
56 checks passed
@zuiderkwast zuiderkwast deleted the revert-psync-after-load-aof branch December 11, 2025 14:57
aradz44 pushed a commit to aradz44/valkey that referenced this pull request Dec 23, 2025
…#2366)" (valkey-io#2925)

This reverts commit 2da21d9.

The implementation in valkey-io#2366 made it possible to perform a partial
resynchronization after loading an AOF file, by storing the replication
offset in the AOF preamble and counting the bytes in the AOF command
stream in the same way as we count byte offset in a replication stream.

However, this approach isn't safe because some commands are replicated
but not stored in the AOF file. This includes the commands REPLCONF,
PING, PUBLISH and module-implemented commands where a module can control
to propagate a command to the replication stream only, to AOF only or to
both. This oversight led to data inconsistency, where the wrong
replication offset is used for partial resynchronization as explained in
issue valkey-io#2904.

The revert caused small merge conflicts with 3c3a196 which are
solved.

Fixes valkey-io#2904.

Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
jdheyburn pushed a commit to jdheyburn/valkey that referenced this pull request Jan 8, 2026
…#2366)" (valkey-io#2925)

This reverts commit 2da21d9.

The implementation in valkey-io#2366 made it possible to perform a partial
resynchronization after loading an AOF file, by storing the replication
offset in the AOF preamble and counting the bytes in the AOF command
stream in the same way as we count byte offset in a replication stream.

However, this approach isn't safe because some commands are replicated
but not stored in the AOF file. This includes the commands REPLCONF,
PING, PUBLISH and module-implemented commands where a module can control
to propagate a command to the replication stream only, to AOF only or to
both. This oversight led to data inconsistency, where the wrong
replication offset is used for partial resynchronization as explained in
issue valkey-io#2904.

The revert caused small merge conflicts with 3c3a196 which are
solved.

Fixes valkey-io#2904.

Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Fix for psync After Restarting AOF-Enabled Instances May Introduce New Data Inconsistency

4 participants