Skip to content

Commit b41795a

Browse files
committed
[SPARK-29087][CORE][STREAMING] Use DelegatingServletContextHandler to avoid CCE
### What changes were proposed in this pull request? [SPARK-27122](#24088) fixes `ClassCastException` at `yarn` module by introducing `DelegatingServletContextHandler`. Initially, this was discovered with JDK9+, but the class path issues affected JDK8 environment, too. After [SPARK-28709](#25439), I also hit the similar issue at `streaming` module. This PR aims to fix `streaming` module by adding `getContextPath` to `DelegatingServletContextHandler` and using it. ### Why are the changes needed? Currently, when we test `streaming` module independently, it fails like the following. ``` $ build/mvn test -pl streaming ... UISeleniumSuite: - attaching and detaching a Streaming tab *** FAILED *** java.lang.ClassCastException: org.sparkproject.jetty.servlet.ServletContextHandler cannot be cast to org.eclipse.jetty.servlet.ServletContextHandler ... Tests: succeeded 337, failed 1, canceled 0, ignored 1, pending 0 *** 1 TEST FAILED *** [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ ``` ### Does this PR introduce any user-facing change? No. ### How was this patch tested? Pass the Jenkins with the modified tests. And do the following manually. Since you can observe this when you run `streaming` module test only (instead of running all), you need to install the changed `core` module and use it. ``` $ java -version openjdk version "1.8.0_222" OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_222-b10) OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.222-b10, mixed mode) $ build/mvn install -DskipTests $ build/mvn test -pl streaming ``` Closes #25791 from dongjoon-hyun/SPARK-29087. Authored-by: Dongjoon Hyun <dhyun@apple.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com> (cherry picked from commit 729b318) Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
1 parent 21649e3 commit b41795a

File tree

2 files changed

+6
-2
lines changed

2 files changed

+6
-2
lines changed

core/src/main/scala/org/apache/spark/ui/WebUI.scala

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -229,4 +229,8 @@ private[spark] class DelegatingServletContextHandler(handler: ServletContextHand
229229
def filterCount(): Int = {
230230
handler.getServletHandler.getFilters.length
231231
}
232+
233+
def getContextPath(): String = {
234+
handler.getContextPath
235+
}
232236
}

streaming/src/test/scala/org/apache/spark/streaming/UISeleniumSuite.scala

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -96,7 +96,7 @@ class UISeleniumSuite
9696

9797
val sparkUI = ssc.sparkContext.ui.get
9898

99-
sparkUI.getHandlers.count(_.getContextPath.contains("/streaming")) should be (5)
99+
sparkUI.getDelegatingHandlers.count(_.getContextPath.contains("/streaming")) should be (5)
100100

101101
eventually(timeout(10 seconds), interval(50 milliseconds)) {
102102
go to (sparkUI.webUrl.stripSuffix("/"))
@@ -197,7 +197,7 @@ class UISeleniumSuite
197197

198198
ssc.stop(false)
199199

200-
sparkUI.getHandlers.count(_.getContextPath.contains("/streaming")) should be (0)
200+
sparkUI.getDelegatingHandlers.count(_.getContextPath.contains("/streaming")) should be (0)
201201

202202
eventually(timeout(10 seconds), interval(50 milliseconds)) {
203203
go to (sparkUI.webUrl.stripSuffix("/"))

0 commit comments

Comments
 (0)