Skip to content

Comments

Run external cluster tests with multiple databases#2176

Merged
zuiderkwast merged 5 commits intovalkey-io:unstablefrom
zuiderkwast:cluster-not-singledb
Jun 11, 2025
Merged

Run external cluster tests with multiple databases#2176
zuiderkwast merged 5 commits intovalkey-io:unstablefrom
zuiderkwast:cluster-not-singledb

Conversation

@zuiderkwast
Copy link
Contributor

@zuiderkwast zuiderkwast commented Jun 4, 2025

Change the runtest flag --cluster-mode to no longer imply --single-db. This allows more tests to run against an external node running in cluster mode.

Also fix incorrectly tagged tests so that ./runtest --singledb passes locally.

Without this change, this doesn't pass because several tests are incorrectly tagged as external:skip instead of singledb:skip or cluster:skip. In the CI, we apparently only run with singledb when implied by --cluster-mode (until now), which we only use in the external cluster mode test. That's why the incorrect tags were not detected.

  • singledb:skip is used for tests that do select and assume that a client has selected db 9.
  • cluster:skip is used when the test uses SWAPDB, cross-slot commands other things forbidden in cluster mode.

Additional fix: End tests with a linebreak when running with the --fastfail flag. A program that doesn't end its output with a linebreak annoys the prompt in the terminal.

Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
Change runtest flag --cluster-mode to no longer imply --single-db. This allows more tests to run against an external node running in cluster mode.

Also fix incorrectly tagged tests so that ./runtest --singledb passes.

Without this change, it doesn't, becuase several tests are incorrectly tagged as external:skip instead of singledb:skip or cluster:skip. Apparently, we only run with --singledb in the CI when we run against an external node, so this was not detected.

The tags are used like this:

* singledb:skip is used for tests that do select and assume that a client has selected db 9.
* cluster:skip is used when the test uses SWAPDB, cross-slot commands other things forbidden in cluster mode.

Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
@zuiderkwast zuiderkwast added the run-extra-tests Run extra tests on this PR (Runs all tests from daily except valgrind and RESP) label Jun 4, 2025
Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
@zuiderkwast zuiderkwast marked this pull request as draft June 5, 2025 00:05
@codecov
Copy link

codecov bot commented Jun 5, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 71.37%. Comparing base (5cc2b25) to head (846cd29).
Report is 7 commits behind head on unstable.

Additional details and impacted files
@@             Coverage Diff              @@
##           unstable    #2176      +/-   ##
============================================
- Coverage     71.50%   71.37%   -0.14%     
============================================
  Files           122      122              
  Lines         66452    66452              
============================================
- Hits          47515    47428      -87     
- Misses        18937    19024      +87     

see 11 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@zuiderkwast zuiderkwast marked this pull request as ready for review June 5, 2025 12:12
@madolson madolson requested a review from xbasel June 10, 2025 03:31
Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
@zuiderkwast zuiderkwast merged commit ac2451b into valkey-io:unstable Jun 11, 2025
60 checks passed
@zuiderkwast zuiderkwast deleted the cluster-not-singledb branch June 11, 2025 07:38
chzhoo pushed a commit to chzhoo/valkey that referenced this pull request Jun 12, 2025
Change the runtest flag `--cluster-mode` to no longer imply
`--single-db`. This allows more tests to run against an external node
running in cluster mode.

Also fix incorrectly tagged tests so that `./runtest --singledb` passes
locally.

Without this change, this doesn't pass because several tests are
incorrectly tagged as external:skip instead of singledb:skip or
cluster:skip. In the CI, we apparently only run with singledb when
implied by `--cluster-mode` (until now), which we only use in the
external cluster mode test. That's why the incorrect tags were not
detected.

* singledb:skip is used for tests that do select and assume that a
client has selected db 9.
* cluster:skip is used when the test uses SWAPDB, cross-slot commands
other things forbidden in cluster mode.

Additional fix: End tests with a linebreak when running with the
`--fastfail` flag. A program that doesn't end its output with a
linebreak annoys the prompt in the terminal.

---------

Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
Signed-off-by: chzhoo <czawyx@163.com>
shanwan1 pushed a commit to shanwan1/valkey that referenced this pull request Jun 13, 2025
Change the runtest flag `--cluster-mode` to no longer imply
`--single-db`. This allows more tests to run against an external node
running in cluster mode.

Also fix incorrectly tagged tests so that `./runtest --singledb` passes
locally.

Without this change, this doesn't pass because several tests are
incorrectly tagged as external:skip instead of singledb:skip or
cluster:skip. In the CI, we apparently only run with singledb when
implied by `--cluster-mode` (until now), which we only use in the
external cluster mode test. That's why the incorrect tags were not
detected.

* singledb:skip is used for tests that do select and assume that a
client has selected db 9.
* cluster:skip is used when the test uses SWAPDB, cross-slot commands
other things forbidden in cluster mode.

Additional fix: End tests with a linebreak when running with the
`--fastfail` flag. A program that doesn't end its output with a
linebreak annoys the prompt in the terminal.

---------

Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
Signed-off-by: shanwan1 <shanwan1@intel.com>
ranshid pushed a commit to ranshid/valkey that referenced this pull request Jun 18, 2025
Change the runtest flag `--cluster-mode` to no longer imply
`--single-db`. This allows more tests to run against an external node
running in cluster mode.

Also fix incorrectly tagged tests so that `./runtest --singledb` passes
locally.

Without this change, this doesn't pass because several tests are
incorrectly tagged as external:skip instead of singledb:skip or
cluster:skip. In the CI, we apparently only run with singledb when
implied by `--cluster-mode` (until now), which we only use in the
external cluster mode test. That's why the incorrect tags were not
detected.

* singledb:skip is used for tests that do select and assume that a
client has selected db 9.
* cluster:skip is used when the test uses SWAPDB, cross-slot commands
other things forbidden in cluster mode.

Additional fix: End tests with a linebreak when running with the
`--fastfail` flag. A program that doesn't end its output with a
linebreak annoys the prompt in the terminal.

---------

Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

run-extra-tests Run extra tests on this PR (Runs all tests from daily except valgrind and RESP)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants