ZSetOperations range*WithScores do not maintain set ordering with a Glide cluster connection

[storance-r7]

I was trying out the pre-release 0.1.0 version, and I noticed the range*WithScores methods on ZSetOperations do not maintain the set's ordering when using a clustered glide connection. These methods do return a set that iterates in the correct order when using a standalone glide connection. The regular range methods (e.g. range, rangeByScore) work correctly for both cluster and standalone connections.

Here is a little snippet that demonstrates the problem:

// clusterConnectionFactory is a ValkeyGlideConnectionFactory setup to connect to a valkey cluster
StringValkeyTemplate template = new StringValkeyTemplate(clusterConnectionFactory);
ZSetOperations<String, String> opsForZSet = template.opsForZSet();

opsForZSet.add("testKey", "value1", 2.0);
opsForZSet.add("testKey", "value2", 1.0);
opsForZSet.add("testKey", "value3", 4.0);
opsForZSet.add("testKey", "value4", 3.0);

// this returns the expected ordering of [value2, value2, value4, value3]
Set<String> rangeResult = opsForZSet.range("testKey", 0, 3);

// this returns what looks like a random ordering
Set<TypedTuple<String>> rangeWithScoresResult = opsForZSet.rangeWithScores("testKey", 0, 3);

I think this is related to this glide client issue. The ZRANGE ... WITHSCORES command returns a Map, and so ClusterValue is treating it as a multi-value. This means ClusterValue ends up calling Map.copyOf() on the result which does not maintain the iteration order of the map being copied.

[ikolomi]

@storance-r7

I have been trying to reproduce the issue with no success - supplementing your code like so:

    @Test
    void testZIssue() {
        try {
            ZSetOperations<String, String> opsForZSet = template.opsForZSet();

            opsForZSet.add("testKey", "value1", 2.0);
            opsForZSet.add("testKey", "value2", 1.0);
            opsForZSet.add("testKey", "value3", 4.0);
            opsForZSet.add("testKey", "value4", 3.0);

            // this returns the expected ordering of [value2, value2, value4, value3]
            Set<String> rangeResult = opsForZSet.range("testKey", 0, 3);

            // this returns what looks like a random ordering
            Set<TypedTuple<String>> rangeWithScoresResult = opsForZSet.rangeWithScores("testKey", 0, 3);
            // print out to see the ordering
            for (TypedTuple<String> tuple : rangeWithScoresResult) {
                System.out.println("Value: " + tuple.getValue() + ", Score: " + tuple.getScore());
            }

        }
        finally {
            template.delete("testKey");
        }
    }

Produces:

Value: value2, Score: 1.0
Value: value1, Score: 2.0
Value: value4, Score: 3.0
Value: value3, Score: 4.0

Which is in correct order.

Could you provide more details such as:

1. What commit id you are using (if building locally) or are you using the package from the maven central?
2. Does it always happen or probabilistic?
3. Which engine are you using?
4. How many shards are in you cluster?
5. Is testKey empty when the code starts

[storance-r7]

1. I used commit 9cf9db1. The one tagged with v0.1.0
2. It does seem probabilistic. We have a few different unit tests in our code around things that use zsets and they would sometimes pass and sometimes fail but I never had a run where every test passed.
3. I was using valkey 8.1.4 with glide client 2.2.0
4. 3 primary shards and no replicas
5. testKey is empty at the start

[jeremyprime]

I was able to reproduce but only under very specific conditions (Spring Data Valkey 0.1.0, GLIDE 2.2.5, cluster, etc), not able to reproduce within a unit test in Spring Data Valkey.

It does seem to be due to GLIDE's map handling in a cluster, although we could also update the affected Spring Data Valkey methods (*WithScores) to sort the values before returning (even if it is redundant once GLIDE is fixed). This could be done in ClusterGlideClientAdapter.customCommand.

[ikolomi]

@jeremyprime Please share the reproduction steps

[jeremyprime]

Spent a little more time and got a unit test working (standalone and cluster, only cluster ordering fails):

    @Test
    void rangeWithScoresStandaloneOrder() {
        StringValkeyTemplate template = new StringValkeyTemplate(ValkeyGlideConnectionFactoryExtension.getConnectionFactory(ValkeyStanalone.class));
        try {
            ZSetOperations<String, String> opsForZSet = template.opsForZSet();

            opsForZSet.add("testKey", "value1", 2.0);
            opsForZSet.add("testKey", "value2", 1.0);
            opsForZSet.add("testKey", "value3", 4.0);
            opsForZSet.add("testKey", "value4", 3.0);

            Set<String> rangeResult = opsForZSet.range("testKey", 0, 3);
            System.out.println("standalone range() result: " + rangeResult);

            String[] rangeArray = rangeResult.toArray(new String[0]);
            assertThat(rangeArray[0]).isEqualTo("value2");
            assertThat(rangeArray[1]).isEqualTo("value1");
            assertThat(rangeArray[2]).isEqualTo("value4");
            assertThat(rangeArray[3]).isEqualTo("value3");

            Set<TypedTuple<String>> rangeWithScoresResult = opsForZSet.rangeWithScores("testKey", 0, 3);
            System.out.println("standalone rangeWithScores() result: " + rangeWithScoresResult);

            TypedTuple<String>[] tuplesArray = rangeWithScoresResult.toArray(new TypedTuple[0]);
            assertThat(tuplesArray[0].getValue()).isEqualTo("value2");
            assertThat(tuplesArray[1].getValue()).isEqualTo("value1");
            assertThat(tuplesArray[2].getValue()).isEqualTo("value4");
            assertThat(tuplesArray[3].getValue()).isEqualTo("value3");
        } finally {
            template.delete("testKey");
        }
    }

    @Test
    @EnabledOnValkeyClusterAvailable
    void rangeWithScoresClusterOrder() {
        StringValkeyTemplate template = new StringValkeyTemplate(ValkeyGlideConnectionFactoryExtension.getConnectionFactory(ValkeyCluster.class));
        try {
            ZSetOperations<String, String> opsForZSet = template.opsForZSet();

            opsForZSet.add("testKey", "value1", 2.0);
            opsForZSet.add("testKey", "value2", 1.0);
            opsForZSet.add("testKey", "value3", 4.0);
            opsForZSet.add("testKey", "value4", 3.0);

            Set<String> rangeResult = opsForZSet.range("testKey", 0, 3);
            System.out.println("cluster range() result: " + rangeResult);

            String[] rangeArray = rangeResult.toArray(new String[0]);
            assertThat(rangeArray[0]).isEqualTo("value2");
            assertThat(rangeArray[1]).isEqualTo("value1");
            assertThat(rangeArray[2]).isEqualTo("value4");
            assertThat(rangeArray[3]).isEqualTo("value3");

            Set<TypedTuple<String>> rangeWithScoresResult = opsForZSet.rangeWithScores("testKey", 0, 3);
            System.out.println("cluster rangeWithScores() result: " + rangeWithScoresResult);

            TypedTuple<String>[] tuplesArray = rangeWithScoresResult.toArray(new TypedTuple[0]);
            assertThat(tuplesArray[0].getValue()).isEqualTo("value2");
            assertThat(tuplesArray[1].getValue()).isEqualTo("value1");
            assertThat(tuplesArray[2].getValue()).isEqualTo("value4");
            assertThat(tuplesArray[3].getValue()).isEqualTo("value3");
        } finally {
            template.delete("testKey");
        }
    }

[jeremyprime]

Waiting on valkey-io/valkey-glide#5257, then after the next GLIDE release which includes that fix will update the GLIDE dependency in Spring Data Valkey and test this bug again.