Skip to content

NETWORKING: Upgrade Netty to 4.1.30 (#34417) #34441

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 6 commits into from
Oct 23, 2018
Merged

NETWORKING: Upgrade Netty to 4.1.30 (#34417) #34441

merged 6 commits into from
Oct 23, 2018

Conversation

original-brownbear
Copy link
Contributor

@original-brownbear original-brownbear commented Oct 15, 2018 •
edited
Loading

Backport PR of #34441 includes backports of memory leak fixes (in test code) that were missing in 6.x (sorry forgot a few when backporting them it seems and now with 4.1.30 they started showing up).

@original-brownbear original-brownbear added :Distributed Coordination/Network Http and internode communication implementations >upgrade labels Oct 15, 2018
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed

@original-brownbear
Copy link
Contributor Author

upgrading 6.x to 4.1.30 causes the following memory leak:

FAILURE 2.33s J0 | Netty4HttpServerTransportTests.testExpectContinueHeader <<< FAILURES!
   > Throwable #1: java.lang.AssertionError: 
   > Expected: an empty collection
   >      but: <[LEAK: ByteBuf.release() was not called before it's garbage-collected. See http://netty.io/wiki/reference-counted-objects.html for more information.
   > Recent access records: 
   > #1:
   > 	io.netty.buffer.AdvancedLeakAwareByteBuf.toString(AdvancedLeakAwareByteBuf.java:742)
   > 	org.elasticsearch.http.netty4.Netty4HttpClient.returnHttpResponseBodies(Netty4HttpClient.java:70)
   > 	org.elasticsearch.http.netty4.Netty4BadRequestTests.testBadParameterEncoding(Netty4BadRequestTests.java:97)
   > 	sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   > 	sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   > 	sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   > 	java.lang.reflect.Method.invoke(Method.java:498)
   > 	com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
   > 	com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
   > 	com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
   > 	com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
   > 	com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   > 	org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
   > 	org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   > 	org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
   > 	org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   > 	org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   > 	com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   > 	com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
   > 	com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
   > 	com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
   > 	com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
   > 	com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
   > 	com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
   > 	com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
   > 	org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   > 	com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   > 	org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
   > 	com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   > 	com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   > 	com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   > 	com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   > 	com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   > 	org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
   > 	org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   > 	org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   > 	org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
   > 	com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   > 	com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
   > 	java.lang.Thread.run(Thread.java:748)
   > Created at:
   > 	io.netty.buffer.PooledByteBufAllocator.newHeapBuffer(PooledByteBufAllocator.java:314)
   > 	io.netty.buffer.AbstractByteBufAllocator.heapBuffer(AbstractByteBufAllocator.java:166)
   > 	io.netty.buffer.AbstractByteBufAllocator.heapBuffer(AbstractByteBufAllocator.java:157)
   > 	io.netty.buffer.CompositeByteBuf.allocBuffer(CompositeByteBuf.java:1661)
   > 	io.netty.buffer.CompositeByteBuf.copy(CompositeByteBuf.java:1340)
   > 	io.netty.buffer.AbstractByteBuf.copy(AbstractByteBuf.java:1190)
   > 	io.netty.buffer.WrappedCompositeByteBuf.copy(WrappedCompositeByteBuf.java:488)
   > 	io.netty.buffer.AdvancedLeakAwareCompositeByteBuf.copy(AdvancedLeakAwareCompositeByteBuf.java:681)
   > 	io.netty.handler.codec.http.HttpObjectAggregator$AggregatedFullHttpResponse.copy(HttpObjectAggregator.java:510)
   > 	org.elasticsearch.http.netty4.Netty4HttpClient$CountDownLatchHandler$1.channelRead0(Netty4HttpClient.java:187)
   > 	org.elasticsearch.http.netty4.Netty4HttpClient$CountDownLatchHandler$1.channelRead0(Netty4HttpClient.java:183)
   > 	io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
   > 	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
   > 	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
   > 	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
   > 	io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
   > 	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
   > 	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
   > 	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
   > 	io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323)
   > 	io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:297)
   > 	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
   > 	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
   > 	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
   > 	io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434)
   > 	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
   > 	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
   > 	io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965)
   > 	io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
   > 	io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644)
   > 	io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544)
   > 	io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498)
   > 	io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458)
   > 	io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:897)
   > 	io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
   > 	java.lang.Thread.run(Thread.java:748)]>
   > 	at __randomizedtesting.SeedInfo.seed([6EB9E80581BB5FB7:415E40733345E35C]:0)
   > 	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
   > 	at org.elasticsearch.test.ESTestCase.checkStaticState(ESTestCase.java:489)
   > 	at org.elasticsearch.test.ESTestCase.after(ESTestCase.java:349)
   > 	at java.lang.Thread.run(Thread.java:748)
  1> [2018-10-15T03:18:22,127][INFO ][o.e.h.n.Netty4HttpServerTransportTests] [testExpectUnsupportedExpectation] before test
  1> [2018-10-15T03:18:22,140][INFO ][o.e.h.n.Netty4HttpServerTransport] [testExpectUnsupportedExpectation] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}
  1> [2018-10-15T03:18:24,468][INFO ][o.e.h.n.Netty4HttpServerTransportTests] [testExpectUnsupportedExpectation] after test
  1> [2018-10-15T03:18:24,472][INFO ][o.e.h.n.Netty4HttpServerTransportTests] [testBindUnavailableAddress] before test
  1> [2018-10-15T03:18:24,487][INFO ][o.e.h.n.Netty4HttpServerTransport] [testBindUnavailableAddress] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}
  1> [2018-10-15T03:18:24,522][INFO ][o.e.h.n.Netty4HttpServerTransportTests] [testBindUnavailableAddress] after test
  1> [2018-10-15T03:18:24,526][INFO ][o.e.h.n.Netty4HttpServerTransportTests] [testBadRequest] before test
  1> [2018-10-15T03:18:24,538][INFO ][o.e.h.n.Netty4HttpServerTransport] [testBadRequest] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}
  1> [2018-10-15T03:18:26,793][INFO ][o.e.h.n.Netty4HttpServerTransportTests] [testBadRequest] after test
  1> [2018-10-15T03:18:26,796][INFO ][o.e.h.n.Netty4HttpServerTransportTests] [testCorsConfigWithDefaults] before test
  1> [2018-10-15T03:18:26,803][INFO ][o.e.h.n.Netty4HttpServerTransportTests] [testCorsConfigWithDefaults] after test
  1> [2018-10-15T03:18:26,806][INFO ][o.e.h.n.Netty4HttpServerTransportTests] [testExpectContinueHeaderContentLengthTooLong] before test
  1> [2018-10-15T03:18:26,816][INFO ][o.e.h.n.Netty4HttpServerTransport] [testExpectContinueHeaderContentLengthTooLong] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}
  1> [2018-10-15T03:18:29,042][INFO ][o.e.h.n.Netty4HttpServerTransportTests] [testExpectContinueHeaderContentLengthTooLong] after test
  1> [2018-10-15T03:18:29,045][INFO ][o.e.h.n.Netty4HttpServerTransportTests] [testDispatchDoesNotModifyThreadContext] before test
  1> [2018-10-15T03:18:29,060][INFO ][o.e.h.n.Netty4HttpServerTransport] [testDispatchDoesNotModifyThreadContext] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}
  1> [2018-10-15T03:18:29,070][INFO ][o.e.h.n.Netty4HttpServerTransportTests] [testDispatchDoesNotModifyThreadContext] after test
  1> [2018-10-15T03:18:29,074][INFO ][o.e.h.n.Netty4HttpServerTransportTests] [testCorsConfig] before test
  1> [2018-10-15T03:18:29,084][INFO ][o.e.h.n.Netty4HttpServerTransportTests] [testCorsConfig] after test
  1> [2018-10-15T03:18:29,088][INFO ][o.e.h.n.Netty4HttpServerTransportTests] [testReadTimeout] before test
  1> [2018-10-15T03:18:29,101][INFO ][o.e.h.n.Netty4HttpServerTransport] [testReadTimeout] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}
  1> [2018-10-15T03:18:31,580][INFO ][o.e.h.n.Netty4HttpServerTransportTests] [testReadTimeout] after test
Completed [12/12] on J0 in 12.84s, 10 tests, 1 failure <<< FAILURES!

on it fixing that here

Tim-Brooks
Tim-Brooks approved these changes Oct 22, 2018
View reviewed changes
Copy link
Contributor

@Tim-Brooks Tim-Brooks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As long as you're happy that you have resolved the CI issues, LGTM

@original-brownbear
Copy link
Contributor Author

@tbrooks8 thanks!

@original-brownbear original-brownbear merged commit 9ce0acf into elastic:6.x Oct 23, 2018
@original-brownbear original-brownbear deleted the 34417-6.x branch October 23, 2018 09:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Tim-Brooks Tim-Brooks Tim-Brooks approved these changes

@jasontedor jasontedor Awaiting requested review from jasontedor

Assignees
No one assigned
Labels
:Distributed Coordination/Network Http and internode communication implementations >upgrade
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@original-brownbear @elasticmachine @Tim-Brooks