Skip to content
This repository has been archived by the owner on Sep 26, 2019. It is now read-only.

Remove reference counting from MessageData #264

Merged
merged 6 commits into from
Nov 18, 2018

Conversation

ajsutton
Copy link
Contributor

PR description

Modifies MessageData so that it's backed by a BytesValue instance instead of a netty ByteBuf. This provides a number of benefits:

  • Removes the need to use reference counting on all MessageData instances thus eliminating an entire category of memory allocation exhaustion bugs. These generated errors like io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 16777216 byte(s) of direct memory (used: 1828716583, max: 1836580864)
  • BytesValue is a simpler interface and immutable so we can significantly reduce the amount of data copying that's required. Previously we copied every incoming message at least 3 times, now it's only a single copy from the netty ByteBuf to a byte[] the BytesValue wraps.
  • Simplifies quite a lot of code by avoiding the need to convert between ByteBuf and ByteValues and removing all the try{} finally {message.release()} blocks.

ajsutton and others added 4 commits November 15, 2018 13:50
MessageData now wraps a BytesValue rather than a Netty ByteBuf so we no longer need to call release/retain and pass that through to the underlying ByteBuf.
Entirely removed support for a BytesValue wrapping a Netty ByteBuf as it does not provide any way to release the underlying ByteBuf.
@ajsutton ajsutton merged commit f869b74 into PegaSysEng:master Nov 18, 2018
@ajsutton ajsutton deleted the release-retain branch November 18, 2018 23:35
@ajsutton ajsutton added the bug Something isn't working label Nov 19, 2018
saltiniroberto added a commit to saltiniroberto/pantheon that referenced this pull request Nov 21, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants