redis_proxy: allow custom commands within transactions

[Kasamix]

Commit Message: redis_proxy: allow custom commands within transactions

Commands configured via :ref:custom_commands are registered with the simple command handler and work as standalone commands, but were rejected with '<command>' command is not supported within transaction when used inside a MULTI/EXEC transaction, because TransactionRequest::create() only checked the static transaction/simple/multi-key command sets. This PR plumbs the configured custom commands into TransactionRequest::create() via a dedicated transaction command handler factory, and accepts them within transactions, treating them as simple commands to match their behavior outside of transactions.

Additional Description:
The command splitter already treats custom commands as simple commands for standalone execution (InstanceImpl registers them with the simple command handler). When a transaction is active, however, all commands are routed to the transaction handler, whose TransactionRequest::create() had no knowledge of the configured custom commands and rejected them. The fix introduces a small TransactionCommandHandlerFactory that holds a reference to the configured custom_commands set and passes it to TransactionRequest::create(), where it is consulted alongside the existing simple/multi-key command set checks. Custom commands inside transactions get the same treatment as simple commands, including the caveat that already applies to multi-key commands in transactions (keys must map to the same upstream).

No runtime guard was added: the change only affects commands explicitly opted in via custom_commands, and only inside transactions, where they previously always failed with a hard error.

Disclosure per the generative AI policy: this change was developed with the assistance of generative AI (Claude Code by Anthropic).

Risk Level: Low — a small bug fix; only affects commands explicitly configured via custom_commands, and only within MULTI/EXEC transactions, where such commands previously always returned an error.

Testing:

• New unit tests in command_splitter_impl_test.cc: custom command forwarded within an active transaction; custom command as the first command of a transaction (MULTI sent upstream first); regression test that a supported-but-not-simple command (scan) is still rejected within a transaction (this rejection path previously had no coverage).
• New integration test PipelinedTransactionWithCustomCommand with a custom_commands-enabled config, reproducing the scenario from the issue (MULTI → json.set → EXEC) end to end.
• All tests under //test/extensions/filters/network/redis_proxy/... and //test/extensions/filters/network/common/redis/... pass locally (macOS arm64). The new tests were verified to fail without the source change.

Docs Changes: Updated the Transactions section of the Redis architecture overview (docs/root/intro/arch_overview/other_protocols/redis.rst) to document that configured custom commands are supported within transactions and treated as single-key commands.

Release Notes: changelogs/current/bug_fixes/redis_proxy__custom-commands-in-transactions.rst

Platform Specific Features: N/A

Fixes #42094

[repokitteh-read-only]

Hi @Kasamix, welcome and thank you for your contribution.

We will try to review your Pull Request as quickly as possible.

In the meantime, please take a look at the contribution guidelines if you have not done so already.

🐱

Caused by: #45958 was opened by Kasamix.

see: more, trace.

[yanavlasov]

/gemini review

[gemini-code-assist]

Warning

Gemini encountered an error creating the review. You can try again by commenting /gemini review.