Skip to content

Comments

Fix server assert on ACL LOAD and resetchannels#3182

Merged
zuiderkwast merged 3 commits intovalkey-io:unstablefrom
dvkashapov:fix-acl-load
Feb 12, 2026
Merged

Fix server assert on ACL LOAD and resetchannels#3182
zuiderkwast merged 3 commits intovalkey-io:unstablefrom
dvkashapov:fix-acl-load

Conversation

@dvkashapov
Copy link
Member

@dvkashapov dvkashapov commented Feb 10, 2026

if the client executing ACL LOAD is a user that is removed from ACL file, it frees itself while still executing the command. Same happens for user while removing it's channel permissions.
This causes c->conn pointer to become null and later serverAssert(c->conn) fails for that client on write in prepareClientToWrite().
Solution is to flag current client to be closed after command completes and reply is written, later this client is freed async.

Bug is present in 8.0, 8.1, 9.0.

Resolves #3181

Signed-off-by: Daniil Kashapov <daniil.kashapov.ykt@gmail.com>
@dvkashapov dvkashapov changed the title Fix server crash on ACL LOAD from deleted user Fix server assert on ACL LOAD from deleted user Feb 10, 2026
@codecov
Copy link

codecov bot commented Feb 10, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.94%. Comparing base (06ccdf9) to head (b6b6ff1).
⚠️ Report is 3 commits behind head on unstable.

Additional details and impacted files
@@             Coverage Diff              @@
##           unstable    #3182      +/-   ##
============================================
+ Coverage     74.92%   74.94%   +0.01%     
============================================
  Files           129      129              
  Lines         71329    71333       +4     
============================================
+ Hits          53441    53457      +16     
+ Misses        17888    17876      -12     
Files with missing lines Coverage Δ
src/acl.c 93.05% <100.00%> (+0.02%) ⬆️

... and 26 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@dvkashapov
Copy link
Member Author

Hi @madolson, can you PTAL at small fix for freeing deleted ACL users from ACL LOAD?

Copy link
Member

@Nikhil-Manglore Nikhil-Manglore left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick fix

Copy link
Contributor

@zuiderkwast zuiderkwast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

Copy link
Member

@madolson madolson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is another crash pathway we should fix. More generally, I wonder if we should do something in the freeClient function to just not free the client if we're currently in a a command?

@zuiderkwast
Copy link
Contributor

More generally, I wonder if we should do something in the freeClient function to just not free the client if we're currently in a a command?

@madolson You mean like this?

void freeClient(c) {
    if (c == current_client) {
        c->flag.close_after_command = 1;
        return;
    }
    ...
}

@madolson
Copy link
Member

More generally, I wonder if we should do something in the freeClient function to just not free the client if we're currently in a a command?

@madolson You mean like this?

void freeClient(c) {
    if (c == current_client) {
        c->flag.close_after_command = 1;
        return;
    }
    ...
}

Effectively yeah. I was tracing through the freeClient. We could maybe also be defensive and just not prepare clients to write that have been freed, instead of asserting. That might also be better than crashing.

@dvkashapov
Copy link
Member Author

I don't think we can do this:

void freeClient(c) {
    if (c == current_client) {
        c->flag.close_after_command = 1;
        return;
    }
    ...
}

Because:

  1. Many callers expect synchronous behavior and the client to actually be freed

  2. If freeClient() is called from: timer callbacks, i/o event handlers, async something then the flag would never be checked and the client would leak memory

  3. Some replication and module specific code won't execute

Maybe we can do this but IDK if this will break something else:

if (c == server.current_client && c->flag.executing_command) {
    c->flag.close_after_command = 1;
    return;
}

Signed-off-by: Daniil Kashapov <daniil.kashapov.ykt@gmail.com>
@madolson
Copy link
Member

If freeClient() is called from: timer callbacks, i/o event handlers, async something then the flag would never be checked and the client would leak memory

These won't be in the context of a command, so server.current_client won't be set. I don't know, I was just hoping there was some better solution :/

Copy link
Member

@madolson madolson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two small suggestions, otherwise it looks good functionality wise.

Co-authored-by: Madelyn Olson <madelyneolson@gmail.com>
Signed-off-by: Daniil Kashapov <daniil.kashapov.ykt@gmail.com>
@dvkashapov dvkashapov changed the title Fix server assert on ACL LOAD from deleted user Fix server assert on ACL LOAD and resetchannels Feb 12, 2026
@zuiderkwast zuiderkwast merged commit a95a75a into valkey-io:unstable Feb 12, 2026
21 of 22 checks passed
@github-project-automation github-project-automation bot moved this to To be backported in Valkey 8.0 Feb 12, 2026
@github-project-automation github-project-automation bot moved this to To be backported in Valkey 8.1 Feb 12, 2026
@github-project-automation github-project-automation bot moved this to To be backported in Valkey 9.0 Feb 12, 2026
@dvkashapov dvkashapov deleted the fix-acl-load branch February 13, 2026 09:14
@roshkhatri roshkhatri moved this from To be backported to 8.1.6 WIP in Valkey 8.1 Feb 17, 2026
roshkhatri pushed a commit to roshkhatri/valkey that referenced this pull request Feb 17, 2026
If the client executing `ACL LOAD` is a user that is removed from ACL
file, it frees itself while still executing the command. Same happens
for user while removing it's channel permissions.
This causes `c->conn` pointer to become `null` and later
`serverAssert(c->conn)` fails for that client on write in
`prepareClientToWrite()`.
Solution is to flag current client to be closed after command completes
and reply is written, later this client is freed async.

Resolves valkey-io#3181

---------

Signed-off-by: Daniil Kashapov <daniil.kashapov.ykt@gmail.com>
Co-authored-by: Madelyn Olson <madelyneolson@gmail.com>
Signed-off-by: Roshan Khatri <rvkhatri@amazon.com>
@roshkhatri roshkhatri moved this from To be backported to 9.0.3 in Valkey 9.0 Feb 17, 2026
roshkhatri pushed a commit to roshkhatri/valkey that referenced this pull request Feb 17, 2026
If the client executing `ACL LOAD` is a user that is removed from ACL
file, it frees itself while still executing the command. Same happens
for user while removing it's channel permissions.
This causes `c->conn` pointer to become `null` and later
`serverAssert(c->conn)` fails for that client on write in
`prepareClientToWrite()`.
Solution is to flag current client to be closed after command completes
and reply is written, later this client is freed async.

Resolves valkey-io#3181

---------

Signed-off-by: Daniil Kashapov <daniil.kashapov.ykt@gmail.com>
Co-authored-by: Madelyn Olson <madelyneolson@gmail.com>
Signed-off-by: Roshan Khatri <rvkhatri@amazon.com>
@roshkhatri roshkhatri moved this from To be backported to 8.0.7 (WIP) in Valkey 8.0 Feb 18, 2026
roshkhatri pushed a commit to roshkhatri/valkey that referenced this pull request Feb 18, 2026
If the client executing `ACL LOAD` is a user that is removed from ACL
file, it frees itself while still executing the command. Same happens
for user while removing it's channel permissions.
This causes `c->conn` pointer to become `null` and later
`serverAssert(c->conn)` fails for that client on write in
`prepareClientToWrite()`.
Solution is to flag current client to be closed after command completes
and reply is written, later this client is freed async.

Resolves valkey-io#3181

---------

Signed-off-by: Daniil Kashapov <daniil.kashapov.ykt@gmail.com>
Co-authored-by: Madelyn Olson <madelyneolson@gmail.com>
roshkhatri pushed a commit to roshkhatri/valkey that referenced this pull request Feb 18, 2026
If the client executing `ACL LOAD` is a user that is removed from ACL
file, it frees itself while still executing the command. Same happens
for user while removing it's channel permissions.
This causes `c->conn` pointer to become `null` and later
`serverAssert(c->conn)` fails for that client on write in
`prepareClientToWrite()`.
Solution is to flag current client to be closed after command completes
and reply is written, later this client is freed async.

Resolves valkey-io#3181

---------

Signed-off-by: Daniil Kashapov <daniil.kashapov.ykt@gmail.com>
Co-authored-by: Madelyn Olson <madelyneolson@gmail.com>
Signed-off-by: Roshan Khatri <rvkhatri@amazon.com>
harrylin98 pushed a commit to harrylin98/valkey_forked that referenced this pull request Feb 19, 2026
If the client executing `ACL LOAD` is a user that is removed from ACL
file, it frees itself while still executing the command. Same happens
for user while removing it's channel permissions.
This causes `c->conn` pointer to become `null` and later
`serverAssert(c->conn)` fails for that client on write in
`prepareClientToWrite()`.
Solution is to flag current client to be closed after command completes
and reply is written, later this client is freed async.

Resolves valkey-io#3181

---------

Signed-off-by: Daniil Kashapov <daniil.kashapov.ykt@gmail.com>
Co-authored-by: Madelyn Olson <madelyneolson@gmail.com>
roshkhatri pushed a commit to roshkhatri/valkey that referenced this pull request Feb 20, 2026
If the client executing `ACL LOAD` is a user that is removed from ACL
file, it frees itself while still executing the command. Same happens
for user while removing it's channel permissions.
This causes `c->conn` pointer to become `null` and later
`serverAssert(c->conn)` fails for that client on write in
`prepareClientToWrite()`.
Solution is to flag current client to be closed after command completes
and reply is written, later this client is freed async.

Resolves valkey-io#3181

---------

Signed-off-by: Daniil Kashapov <daniil.kashapov.ykt@gmail.com>
Co-authored-by: Madelyn Olson <madelyneolson@gmail.com>
Signed-off-by: Roshan Khatri <rvkhatri@amazon.com>
roshkhatri pushed a commit to roshkhatri/valkey that referenced this pull request Feb 20, 2026
If the client executing `ACL LOAD` is a user that is removed from ACL
file, it frees itself while still executing the command. Same happens
for user while removing it's channel permissions.
This causes `c->conn` pointer to become `null` and later
`serverAssert(c->conn)` fails for that client on write in
`prepareClientToWrite()`.
Solution is to flag current client to be closed after command completes
and reply is written, later this client is freed async.

Resolves valkey-io#3181

---------

Signed-off-by: Daniil Kashapov <daniil.kashapov.ykt@gmail.com>
Co-authored-by: Madelyn Olson <madelyneolson@gmail.com>
Signed-off-by: Roshan Khatri <rvkhatri@amazon.com>
madolson added a commit that referenced this pull request Feb 24, 2026
If the client executing `ACL LOAD` is a user that is removed from ACL
file, it frees itself while still executing the command. Same happens
for user while removing it's channel permissions.
This causes `c->conn` pointer to become `null` and later
`serverAssert(c->conn)` fails for that client on write in
`prepareClientToWrite()`.
Solution is to flag current client to be closed after command completes
and reply is written, later this client is freed async.

Resolves #3181

---------

Signed-off-by: Daniil Kashapov <daniil.kashapov.ykt@gmail.com>
Co-authored-by: Madelyn Olson <madelyneolson@gmail.com>
Signed-off-by: Roshan Khatri <rvkhatri@amazon.com>
madolson added a commit that referenced this pull request Feb 24, 2026
If the client executing `ACL LOAD` is a user that is removed from ACL
file, it frees itself while still executing the command. Same happens
for user while removing it's channel permissions.
This causes `c->conn` pointer to become `null` and later
`serverAssert(c->conn)` fails for that client on write in
`prepareClientToWrite()`.
Solution is to flag current client to be closed after command completes
and reply is written, later this client is freed async.

Resolves #3181

---------

Signed-off-by: Daniil Kashapov <daniil.kashapov.ykt@gmail.com>
Co-authored-by: Madelyn Olson <madelyneolson@gmail.com>
Signed-off-by: Roshan Khatri <rvkhatri@amazon.com>
madolson added a commit that referenced this pull request Feb 24, 2026
If the client executing `ACL LOAD` is a user that is removed from ACL
file, it frees itself while still executing the command. Same happens
for user while removing it's channel permissions.
This causes `c->conn` pointer to become `null` and later
`serverAssert(c->conn)` fails for that client on write in
`prepareClientToWrite()`.
Solution is to flag current client to be closed after command completes
and reply is written, later this client is freed async.

Resolves #3181

---------

Signed-off-by: Daniil Kashapov <daniil.kashapov.ykt@gmail.com>
Co-authored-by: Madelyn Olson <madelyneolson@gmail.com>
Signed-off-by: Roshan Khatri <rvkhatri@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: 8.0.7 (WIP)
Status: 8.1.6 WIP
Status: 9.0.3 (WIP)
Status: Done

Development

Successfully merging this pull request may close these issues.

[CRASH] Crash after ACL LOAD under removed user

4 participants