Python client serializes paging preciate differently than Java and .Net causing 5.4 compatibility breaks

[srknzl]

Py client serializes the paging predicate by putting it in itself (definitely wrong but its tests work somehow) See

hazelcast-python-client/hazelcast/protocol/__init__.py

Line 133 in f32c630

predicate_data = to_data(predicate)

While other clients serialize the actual internal (non paging) predicate inside it (correctly) see https://github.com/hazelcast/hazelcast-mono/blob/df42d6a1c7666263f3fc9bb47765c850c3986321/hazelcast/hazelcast/src/main/java/com/hazelcast/client/impl/protocol/codec/holder/PagingPredicateHolder.java#L168

The error we got is:

hazelcast.errors.HazelcastSerializationError: Exception from server: com.hazelcast.nio.serialization.HazelcastSerializationException: Problem while reading DataSerializable, namespace: -20, ID: 15, class: 'null', exception: offset 41, count 1447119957, length 46
E            	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.rethrowReadException(DataSerializableSerializer.java:190)
E           	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.readInternal(DataSerializableSerializer.java:171)
E           	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:113)
E           	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:52)
E           	at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:44)
E           	at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toObject(AbstractSerializationService.java:271)
E           	at com.hazelcast.client.impl.protocol.codec.holder.PagingPredicateHolder.lambda$asPredicate$0(PagingPredicateHolder.java:108)
E           	at com.hazelcast.internal.namespace.impl.NoOpNamespaceService.callWithNamespace(NoOpNamespaceService.java:87)
E           	at com.hazelcast.internal.namespace.NamespaceUtil.callWithNamespace(NamespaceUtil.java:128)
E           	at com.hazelcast.internal.namespace.NamespaceUtil.callWithNamespace(NamespaceUtil.java:118)
E           	at com.hazelcast.client.impl.protocol.codec.holder.PagingPredicateHolder.asPredicate(PagingPredicateHolder.java:106)
E           	at com.hazelcast.client.impl.protocol.task.map.AbstractMapQueryWithPagingPredicateMessageTask.getPredicate(AbstractMapQueryWithPagingPredicateMessageTask.java:63)
E           	at com.hazelcast.client.impl.protocol.task.map.AbstractMapQueryMessageTask.call(AbstractMapQueryMessageTask.java:105)
E           	at com.hazelcast.client.impl.protocol.task.AbstractCallableMessageTask.processMessage(AbstractCallableMessageTask.java:35)
E           	at com.hazelcast.client.impl.protocol.task.AbstractMessageTask.initializeAndProcessMessage(AbstractMessageTask.java:166)
E           	at com.hazelcast.client.impl.protocol.task.AbstractMessageTask.run(AbstractMessageTask.java:129)
E           	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
E           	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
E           	at java.lang.Thread.run(Thread.java:829)
E           	at com.hazelcast.internal.util.executor.HazelcastManagedThread.executeRun(HazelcastManagedThread.java:76)
E           	at com.hazelcast.internal.util.executor.PoolExecutorThreadFactory$ManagedThread.executeRun(PoolExecutorThreadFactory.java:74)
E           	at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:111)
E           Caused by: Exception from server: java.lang.StringIndexOutOfBoundsException: offset 41, count 1447119957, length 46
E            	at java.lang.String.checkBoundsOffCount(String.java:3304)
E           	at java.lang.String.<init>(String.java:505)
E           	at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataInput.readUTFInternal(ByteArrayObjectDataInput.java:720)
E           	at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataInput.readString(ByteArrayObjectDataInput.java:594)
E           	at com.hazelcast.query.impl.predicates.PagingPredicateImpl.readData(PagingPredicateImpl.java:398)
E           	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.readInternal(DataSerializableSerializer.java:167)
E           	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:113)
E           	at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:52)
E           	at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:44)
E           	at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toObject(AbstractSerializationService.java:271)
E           	at com.hazelcast.client.impl.protocol.codec.holder.PagingPredicateHolder.lambda$asPredicate$0(PagingPredicateHolder.java:108)
E           	at com.hazelcast.internal.namespace.impl.NoOpNamespaceService.callWithNamespace(NoOpNamespaceService.java:87)
E           	at com.hazelcast.internal.namespace.NamespaceUtil.callWithNamespace(NamespaceUtil.java:128)
E           	at com.hazelcast.internal.namespace.NamespaceUtil.callWithNamespace(NamespaceUtil.java:118)
E           	at com.hazelcast.client.impl.protocol.codec.holder.PagingPredicateHolder.asPredicate(PagingPredicateHolder.java:106)
E           	at com.hazelcast.client.impl.protocol.task.map.AbstractMapQueryWithPagingPredicateMessageTask.getPredicate(AbstractMapQueryWithPagingPredicateMessageTask.java:63)
E           	at com.hazelcast.client.impl.protocol.task.map.AbstractMapQueryMessageTask.call(AbstractMapQueryMessageTask.java:105)
E           	at com.hazelcast.client.impl.protocol.task.AbstractCallableMessageTask.processMessage(AbstractCallableMessageTask.java:35)
E           	at com.hazelcast.client.impl.protocol.task.AbstractMessageTask.initializeAndProcessMessage(AbstractMessageTask.java:166)
E           	at com.hazelcast.client.impl.protocol.task.AbstractMessageTask.run(AbstractMessageTask.java:129)
E           	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
E           	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
E           	at java.lang.Thread.run(Thread.java:829)
E           	at com.hazelcast.internal.util.executor.HazelcastManagedThread.executeRun(HazelcastManagedThread.java:76)
E           	at com.hazelcast.internal.util.executor.PoolExecutorThreadFactory$ManagedThread.executeRun(PoolExecutorThreadFactory.java:74)
E           	at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:111)
E           Caused by: java.lang.StringIndexOutOfBoundsException

on the test tests/integration/backward_compatible/predicate_test.py:398 with OS 5.0.1 client with OS server on hz commit 5142e52 (5.4.0-BETA-1) https://github.com/hazelcast/client-compatibility-suites/actions/runs/7273653696/job/19818389435

The correct fix in my opinion is that releasing a new patch for all python clients that’ll work with 5.4 hz and onwards.