[SPARK-12784][UI]Fix Spark UI IndexOutOfBoundsException with dynamic allocation

[zsxwing]

Add listener.synchronized to get storageStatusList and execInfo atomically.

[tgravescs]

thanks for picking this up so quickly.
I need to look at this a bit further, so I assume the other end modifying the listener.storageStatusList is also synchronized?

One concern I have here is that I don't want to affect application performance just because someone is viewing the UI. If the UI load blocks the application from running, that is bad. I would rather see us get a read only copy of it or something then synchronize.

I'll try to look in more detail tomorrow.

[zsxwing]

It usually doesn't affect your application performance. It only blocks the listener bus thread when rendering a page.

[SparkQA]

Test build #49269 has finished for PR 10728 at commit 0c6d0c5.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[zsxwing]

CC @andrewor14

[andrewor14]

@sarutak

[andrewor14]

This looks OK, and yeah we're not blocking any thread that's actually doing the scheduling or execution so it should be fine.

[andrewor14]

I would add a comment to explain why we need to do this in synchronized though, and include issue number

[sarutak]

LGTM

[SparkQA]

Test build #49343 has finished for PR 10728 at commit b47a574.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[sarutak]

I found a needless semi-colon in ExecutorsPage.scala.

https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/ui/exec/ExecutorsPage.scala#L110

But it's not a big deal and we have already passed SparkQA so I think it's OK to merge without removing the semi-colon.

[zsxwing]

Thanks, merging to master, 1.6 and 1.5