Commit b268455
[SPARK-8437] [DOCS] Using directory path without wildcard for filename slow for large number of files with wholeTextFiles and binaryFiles
Note that 'dir/*' can be more efficient in some Hadoop FS implementations that 'dir/'
Author: Sean Owen <sowen@cloudera.com>
Closes apache#7036 from srowen/SPARK-8437 and squashes the following commits:
0e813ae [Sean Owen] Note that 'dir/*' can be more efficient in some Hadoop FS implementations that 'dir/'
(cherry picked from commit 5d30eae)
Signed-off-by: Andrew Or <andrew@databricks.com>1 parent cdfa388 commit b268455
1 file changed
+6
-2
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
824 | 824 | | |
825 | 825 | | |
826 | 826 | | |
| 827 | + | |
| 828 | + | |
827 | 829 | | |
828 | 830 | | |
829 | 831 | | |
| |||
871 | 873 | | |
872 | 874 | | |
873 | 875 | | |
874 | | - | |
875 | | - | |
876 | 876 | | |
| 877 | + | |
| 878 | + | |
| 879 | + | |
| 880 | + | |
877 | 881 | | |
878 | 882 | | |
879 | 883 | | |
| |||
0 commit comments