Skip to content

Investigate performance improvement in java MessagePack deserializer #25089

Open
@wtgodbe

Description

@wtgodbe

In the below block, we effectively deserialize maps/arrays twice - first to find out how long they are, second to feed that number of bytes into ObjectMapper.readValue. We should investigate if its possible to pass the bytes in from their starting offset, and to find out after the fact how many bytes were read so we can advance the underlying buffer. Perhaps with a custom Reader that keeps track of how many bytes it has read?

case ARRAY:
readBytesStart = unpacker.getTotalReadBytes();
length = unpacker.unpackArrayHeader();
for (int i = 0; i < length; i++) {
readValue(unpacker, Object.class, payload, false);
}
if (outermostCall) {
// Check how many bytes we've read, grab that from the payload, and deserialize with objectMapper
byte[] payloadBytes = payload.array();
// If itemType was null, we were just in this method to advance the buffer. return null.
if (itemType == null) {
return null;
}
return objectMapper.readValue(payloadBytes, payload.position() + (int) readBytesStart, (int) (unpacker.getTotalReadBytes() - readBytesStart),
typeFactory.constructType(itemType));
} else {
// This is an inner call to readValue - we just need to read the right number of bytes
// We can return null, and the outermost call will know how many bytes to give to objectMapper.
return null;
}
case MAP:
readBytesStart = unpacker.getTotalReadBytes();
length = unpacker.unpackMapHeader();
for (int i = 0; i < length; i++) {
readValue(unpacker, Object.class, payload, false);
readValue(unpacker, Object.class, payload, false);
}
if (outermostCall) {
// Check how many bytes we've read, grab that from the payload, and deserialize with objectMapper
byte[] payloadBytes = payload.array();
byte[] mapBytes = Arrays.copyOfRange(payloadBytes, payload.position() + (int) readBytesStart,
payload.position() + (int) unpacker.getTotalReadBytes());
// If itemType was null, we were just in this method to advance the buffer. return null.
if (itemType == null) {
return null;
}
return objectMapper.readValue(payloadBytes, payload.position() + (int) readBytesStart, (int) (unpacker.getTotalReadBytes() - readBytesStart),
typeFactory.constructType(itemType));
} else {
// This is an inner call to readValue - we just need to read the right number of bytes
// We can return null, and the outermost call will know how many bytes to give to objectMapper.
return null;
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    affected-very-fewThis issue impacts very few customersarea-signalrIncludes: SignalR clients and serversenhancementThis issue represents an ask for new feature or an enhancement to an existing oneseverity-minorThis label is used by an internal tool

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions