You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
if (registration.isTypeNameAscii())
output.writeAscii(type.getName());
elseoutput.writeString(type.getName());
in other words Kryo writes a name of the class that is generated (something like $Configuration$$EnhancerByCGLIB$$d0595dcf), later it finds registered serializer and serializes cglibproxy object via registered serializer.
The problem happens when the other side attempts deserialization: there is no class named like $Configuration$$EnhancerByCGLIB$$d0595dcf as it was generated in a different JVM.
One way to go around this problem could be to override ClassResolver (since all of classresolvers extend DefaultClassResolver) and provide custom implementation of writeName .
Providing custom CollectionSerializer does not look like a viable option (see EsotericSoftware/kryo#943 for an attempt to do something similar).
Now, Kryo allows us to pass custom ClassResolver, but KryoSerializer is quite opinionated (because it provides custom classresolvers), so I had to resort to reflection:
// inside KryoInit extends DefaultKryoInitializer {valclassResolver= kryo.getClassResolver
classResolver match {
caseresolver: DefaultClassResolver=>// I really did not want to use reflection, but alternative is to re-implement/copy KryoSerializer (from altoo) in order to use custom classresolver
replaceClassResolver(kryo, resolver)
case _ =>thrownewIllegalStateException(s"Unable to replace unknown classResolver type ${classResolver.getClass.getName}")
}
}
privatedefreplaceClassResolver(kryo: ScalaKryo, original: DefaultClassResolver):Unit= {
// Kryo might use CollectionSerializer under the hood to serialize "syntheticProperties" collection.// If this happens it will write name of the class and then serialize it using registered serializer// If collection element is a cglib proxy it means that the class was generated only on the node where serialization happens// This is a problem because other nodes in the cluster do not know about this custom class - those nodes do have serializer// and can deserialize a reference into their own proxy.// That's why we want to tell Kryo that it should use "ResolvableLazyConfigurationItem" as a class name instead of the one created by cglib.// Use of KryoSerializer and Akka make this difficult to test on a single node and with cluster you have to debug through Kryo codebase// to see which serializers and class names are being picked up.valkryoClazz=classOf[Kryo]
valclassResolverField= kryoClazz.getDeclaredField("classResolver")
classResolverField.setAccessible(true)
valxlrClassResolver=newXlrClassResolver(original)
classResolverField.set(kryo, xlrClassResolver)
xlrClassResolver.setKryo(kryo)
}
I would like to avoid reflection and directly extend classresolvers used by KryoSerializer and pass custom class resolver (which should extend "expected" classresolver type for particular akka-kryo configuration) via config to akka-kryo-serialization.
and then io.altoo.akka.serialization.kryo.KryoSerializer#getKryo should be a bit more complicated as it would have to check if there is a custom class-resolver and if it is extending expected class resolver.
Alternatively, just make getKryo protected so we can extend KryoSerializer and provide our own implementation of getKryo.
The text was updated successfully, but these errors were encountered:
Sorry, this kind of went under the radar.
I guess the option to provide a class-resolver via config would make sense for such scenarios, but the class resolver chosen depends on other settings:
val classResolver =
if (settings.idStrategy == "incremental") new KryoClassResolver(settings.implicitRegistrationLogging)
else if (settings.resolveSubclasses) new SubclassResolver()
else new DefaultClassResolver()
Hi, let me explain original issue first:
kryo.writeClassAndObject
useswriteClass(output, object.getClass());
and writes generated class name:ClassResolver
writes a class name:writeName
method uses:in other words Kryo writes a name of the class that is generated (something like
$Configuration$$EnhancerByCGLIB$$d0595dcf
), later it finds registered serializer and serializes cglibproxy object via registered serializer.The problem happens when the other side attempts deserialization: there is no class named like
$Configuration$$EnhancerByCGLIB$$d0595dcf
as it was generated in a different JVM.One way to go around this problem could be to override ClassResolver (since all of classresolvers extend DefaultClassResolver) and provide custom implementation of writeName .
Providing custom CollectionSerializer does not look like a viable option (see EsotericSoftware/kryo#943 for an attempt to do something similar).
Now, Kryo allows us to pass custom ClassResolver, but KryoSerializer is quite opinionated (because it provides custom classresolvers), so I had to resort to reflection:
and then again in custom classresolver:
I would like to avoid reflection and directly extend classresolvers used by KryoSerializer and pass custom class resolver (which should extend "expected" classresolver type for particular akka-kryo configuration) via config to akka-kryo-serialization.
Something like:
and then
io.altoo.akka.serialization.kryo.KryoSerializer#getKryo
should be a bit more complicated as it would have to check if there is a custom class-resolver and if it is extending expected class resolver.Alternatively, just make getKryo protected so we can extend KryoSerializer and provide our own implementation of getKryo.
The text was updated successfully, but these errors were encountered: