[SPARK-9326] Close lock file used for file downloads.

[kayousterhout]

A lock file is used to ensure multiple executors running on the
same machine don't download the same file concurrently. Spark never
closes these lock files (releasing the lock does not close the
underlying file); this commit fixes that.

cc @vanzin (looks like you've been involved in various other fixes surrounding these lock files)

[vanzin]

LGTM (I assume closing the channel also closes the RAF from which is was created).

[vanzin]

(Yes, just double-checked that's the behavior for closing the channel.)

[SparkQA]

Test build #38398 has finished for PR 7650 at commit 0401bd1.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[kayousterhout]

Jenkins, retest this please

[SparkQA]

Test build #95 has finished for PR 7650 at commit 0401bd1.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[kayousterhout]

Jenkins, retest this please

[SparkQA]

Test build #38407 has finished for PR 7650 at commit 0401bd1.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • abstract class InternalRow extends Serializable
  • class GenericRow(protected[sql] val values: Array[Any]) extends Row
  • class GenericInternalRow(protected[sql] val values: Array[Any]) extends InternalRow
  • class GenericInternalRowWithSchema(values: Array[Any], val schema: StructType)
  • class GenericMutableRow(val values: Array[Any]) extends MutableRow

[SparkQA]

Test build #96 has finished for PR 7650 at commit 0401bd1.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #38409 timed out for PR 7650 at commit 0401bd1 after a configured wait of 175m.

[kayousterhout]

Jenkins, retest this please

[SparkQA]

Test build #38419 has finished for PR 7650 at commit 0401bd1.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #98 has finished for PR 7650 at commit 0401bd1.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[JoshRosen]

LGTM as well, so I'm going to merge this to master and branch-1.4. Thanks!

[kayousterhout]

Thanks @JoshRosen!