[SPARK-35002][INFRA][FOLLOW-UP] Use localhost instead of 127.0.0.1 at SPARK_LOCAL_IP in GA builds #32102
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
wangyum
approved these changes
Apr 9, 2021
|
Kubernetes integration test starting |
|
Kubernetes integration test status failure |
|
Merged to master. |
|
Test build #137118 has finished for PR 32102 at commit
|
|
Thanks! |
pan3793
pushed a commit
to apache/kyuubi
that referenced
this pull request
Apr 9, 2021
… testing with Github Action  [](https://github.com/yaooqinn/kyuubi/pull/503)     [<img width="16" alt="Powered by Pull Request Badge" src="https://user-images.githubusercontent.com/1393946/111216524-d2bb8e00-85d4-11eb-821b-ed4c00989c02.png">](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info)<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT --> <!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html 2. If the PR is related to an issue in https://github.com/yaooqinn/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'. 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'. --> ### _Why are the changes needed?_ Refer apache/spark#32096, apache/spark#32102, this PR tries to fix the java.net.BindException when testing with Github Action. ``` SparkOperationSuite: *** RUN ABORTED *** java.net.BindException: Cannot assign requested address: Service 'sparkDriver' failed after 16 retries (on a random free port)! Consider explicitly setting the appropriate binding address for the service 'sparkDriver' (for example spark.driver.bindAddress for SparkDriver) to the correct binding address. at sun.nio.ch.Net.bind0(Native Method) at sun.nio.ch.Net.bind(Net.java:461) at sun.nio.ch.Net.bind(Net.java:453) at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:222) at io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:134) at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:550) at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1334) at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:506) at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:491) at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:973) ``` Also transfer FRONTEND_BIND_HOST by connection string to fix similar issue. ``` Cause: java.lang.RuntimeException: org.apache.kyuubi.KyuubiSQLException:org.apache.kyuubi.KyuubiException: Failed to initialize frontend service on fv-az207-19/10.1.1.0:0. at org.apache.kyuubi.service.FrontendService.initialize(FrontendService.scala:102) at org.apache.kyuubi.service.CompositeService.$anonfun$initialize$1(CompositeService.scala:40) at org.apache.kyuubi.service.CompositeService.$anonfun$initialize$1$adapted(CompositeService.scala:40) at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62) at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49) at org.apache.kyuubi.service.CompositeService.initialize(CompositeService.scala:40) at org.apache.kyuubi.service.Serverable.initialize(Serverable.scala:44) at org.apache.kyuubi.engine.spark.SparkSQLEngine.initialize(SparkSQLEngine.scala:49) at org.apache.kyuubi.engine.spark.SparkSQLEngine$.startEngine(SparkSQLEngine.scala:105) at org.apache.kyuubi.engine.spark.SparkSQLEngine$.main(SparkSQLEngine.scala:118) at org.apache.kyuubi.engine.spark.SparkSQLEngine.main(SparkSQLEngine.scala) ``` ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/latest/tools/testing.html#running-tests) locally before make a pull request Closes #503 from turboFei/KYUUBI-502. Closes #502 1b10253 [fwang12] use localhost instead of 127.0.0.1 c104ce3 [fwang12] address comments 1e549c1 [fwang12] revert install shade 457ce2f [fwang12] try set frontend bind host in connection string 8bcd5a0 [fwang12] revert env KYUUBI_FRONTEND_BIND_HOST and set kyuubi.frontend.bind.host to 127.0.0.1 in scalatest-maven-plugin 717a992 [fwang12] update doc d5ba05a [fwang12] add install shaded jars in release.yml e8b2372 [fwang12] involve KYUUBI_FRONTEND_BIND_HOST 5eb7cdb [fwang12] also set KYUUBI_FRONTEND_BIND_HOST env to 127.0.0.1 7d70819 [fwang12] [KYUUBI #502][SPARK-35002][INFRA] Fix the java.net.BindException when testing with Github Action Authored-by: fwang12 <fwang12@ebay.com> Signed-off-by: Cheng Pan <379377944@qq.com>
|
I merged to branch-3.1 and branch-3.0 too. Seems like it has the same issue. |
HyukjinKwon
added a commit
that referenced
this pull request
Apr 14, 2021
… SPARK_LOCAL_IP in GA builds This PR replaces 127.0.0.1 to `localhost`. - #32096 (comment) - #32096 (comment) No, dev-only. I didn't test it because it's CI specific issue. I will test it in Github Actions build in this PR. Closes #32102 from HyukjinKwon/SPARK-35002. Authored-by: HyukjinKwon <gurwls223@apache.org> Signed-off-by: Yuming Wang <yumwang@ebay.com> (cherry picked from commit a3d1e00) Signed-off-by: HyukjinKwon <gurwls223@apache.org>
HyukjinKwon
added a commit
that referenced
this pull request
Apr 14, 2021
… SPARK_LOCAL_IP in GA builds This PR replaces 127.0.0.1 to `localhost`. - #32096 (comment) - #32096 (comment) No, dev-only. I didn't test it because it's CI specific issue. I will test it in Github Actions build in this PR. Closes #32102 from HyukjinKwon/SPARK-35002. Authored-by: HyukjinKwon <gurwls223@apache.org> Signed-off-by: Yuming Wang <yumwang@ebay.com> (cherry picked from commit a3d1e00) Signed-off-by: HyukjinKwon <gurwls223@apache.org>
|
Just in case it is useful, I created an alternative fix which fixes the root cause of the problem. Reverse name lookups for the current hostname is broken in GitHub Actions Runner VMs (reported as actions/runner-images#3185) and this is the workaround: #32173 . |
flyrain
pushed a commit
to flyrain/spark
that referenced
this pull request
Sep 21, 2021
… SPARK_LOCAL_IP in GA builds This PR replaces 127.0.0.1 to `localhost`. - apache#32096 (comment) - apache#32096 (comment) No, dev-only. I didn't test it because it's CI specific issue. I will test it in Github Actions build in this PR. Closes apache#32102 from HyukjinKwon/SPARK-35002. Authored-by: HyukjinKwon <gurwls223@apache.org> Signed-off-by: Yuming Wang <yumwang@ebay.com> (cherry picked from commit a3d1e00) Signed-off-by: HyukjinKwon <gurwls223@apache.org>
<!--](https://user-images.githubusercontent.com/1393946/111216524-d2bb8e00-85d4-11eb-821b-ed4c00989c02.png">](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info)<!--){kind=link}
31 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
This PR replaces 127.0.0.1 to
localhost.Why are the changes needed?
Does this PR introduce any user-facing change?
No, dev-only.
How was this patch tested?
I didn't test it because it's CI specific issue. I will test it in Github Actions build in this PR.