Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Spark-20812][Mesos] Add secrets support to the dispatcher #18837

Closed
Closed
Changes from 1 commit
Commits
Show all changes
2591 commits
Select commit Hold shift + click to select a range
7d7eb0a
[SPARK-21223] Change fileToAppInfo in FsHistoryProvider to fix concur…
Jun 30, 2017
c633181
[SPARK-21129][SQL] Arguments of SQL function call should not be named…
gatorsmile Jun 30, 2017
e33e95b
[SPARK-21052][SQL][FOLLOW-UP] Add hash map metrics to join
viirya Jun 30, 2017
f44b8bb
[SPARK-17528][SQL] data should be copied properly before saving into …
cloud-fan Jul 1, 2017
fd968d3
[SPARK-21127][SQL] Update statistics after data changing commands
wzhfy Jul 1, 2017
14bb486
[SPARK-21273][SQL] Propagate logical plan stats using visitor pattern…
rxin Jul 1, 2017
81acc79
[SPARK-21275][ML] Update GLM test to use supportedFamilyNames
actuaryzhang Jul 1, 2017
1dda01a
[SPARK-18518][ML] HasSolver supports override
zhengruifeng Jul 1, 2017
d0bb8a8
[SPARK-21170][CORE] Utils.tryWithSafeFinallyAndFailureCallbacks throw…
Jul 1, 2017
57e5008
[SPARK-21260][SQL][MINOR] Remove the unused OutputFakerExec
jiangxb1987 Jul 2, 2017
b99d6ad
[SPARK-19852][PYSPARK][ML] Python StringIndexer supports 'keep' to ha…
yanboliang Jul 2, 2017
94899a1
[SPARK-18004][SQL] Make sure the date or timestamp related predicate …
SharpRay Jul 3, 2017
ccd8bfb
[SPARK-21250][WEB-UI] Add a url in the table of 'Running Executors' i…
Jul 3, 2017
e8d3dbb
[SPARK-21137][CORE] Spark reads many small files slowly
srowen Jul 3, 2017
2583529
[TEST] Load test table based on case sensitivity
wzhfy Jul 3, 2017
de6831b
[SPARK-21102][SQL] Refresh command is too aggressive in parsing
Jul 3, 2017
9175f9b
[SPARK-20073][SQL] Prints an explicit warning message in case of NULL…
maropu Jul 3, 2017
6245541
[SPARK-21284][SQL] rename SessionCatalog.registerFunction parameter name
cloud-fan Jul 3, 2017
bec371a
[TEST] Different behaviors of SparkContext Conf when building SparkSe…
gatorsmile Jul 3, 2017
8f75802
[SPARK-21283][CORE] FileOutputStream should be created as append mode
10110346 Jul 4, 2017
f206ed0
[SPARK-21264][PYTHON] Call cross join path in join without 'on' and w…
HyukjinKwon Jul 4, 2017
17f6307
[MINOR] Add french stop word "les"
ebuildy Jul 4, 2017
17c0952
[MINOR][SPARK SUBMIT] Print out R file usage in spark-submit
HyukjinKwon Jul 4, 2017
b1cab7e
[SPARK-19507][SPARK-21296][PYTHON] Avoid per-record type dispatch in …
HyukjinKwon Jul 4, 2017
f3910ad
[SPARK-21256][SQL] Add withSQLConf to Catalyst Test
gatorsmile Jul 4, 2017
dad8be9
[SPARK-19726][SQL] Faild to insert null timestamp value to mysql usin…
Jul 4, 2017
04e7036
[SPARK-20256][SQL] SessionState should be created more lazily
dongjoon-hyun Jul 4, 2017
72f0e6d
[SPARK-21268][MLLIB] Move center calculations to a distributed map in…
gjgd Jul 4, 2017
1e8d185
[SPARK-20889][SPARKR] Grouped documentation for WINDOW column methods
actuaryzhang Jul 4, 2017
883ee9d
[MINOR][SPARKR] ignore Rplots.pdf test output after running R tests
wangmiao1981 Jul 4, 2017
0369d8e
[SPARK-21295][SQL] Use qualified names in error message for missing r…
gatorsmile Jul 5, 2017
ce7c457
[SPARK-21300][SQL] ExternalMapToCatalyst should null-check map key pr…
ueshin Jul 5, 2017
cc327a7
[SPARK-20889][SPARKR][FOLLOWUP] Clean up grouped doc for column methods
actuaryzhang Jul 5, 2017
ee5f437
[SPARK-21304][SQL] remove unnecessary isNull variable for collection …
cloud-fan Jul 5, 2017
93061c9
[SPARK-18623][SQL] Add `returnNullable` to `StaticInvoke` and modify …
ueshin Jul 5, 2017
c44dd51
[SPARK-21310][ML][PYSPARK] Expose offset in PySpark
actuaryzhang Jul 5, 2017
3e60870
[SPARK-16167][SQL] RowEncoder should preserve array/map type nullabil…
ueshin Jul 5, 2017
5589ded
[SPARK-20383][SQL] Supporting Create [temporary] Function with the ke…
Jul 5, 2017
4067ef0
[SPARK-21286][TEST] Modified StorageTabSuite unit test
Geek-He Jul 5, 2017
a9e2cd5
[SPARK-20858][DOC][MINOR] Document ListenerBus event queue size
sadikovi Jul 5, 2017
975e5e2
[SPARK-19439][PYSPARK][SQL] PySpark's registerJavaFunction Should Sup…
zjffdu Jul 5, 2017
9ddb875
[SPARK-21307][SQL] Remove SQLConf parameters from the parser-related …
gatorsmile Jul 5, 2017
f530b93
[SPARK-21278][PYSPARK] Upgrade to Py4J 0.10.6
dongjoon-hyun Jul 5, 2017
9da4476
[SPARK-21248][SS] The clean up codes in StreamExecution should not be…
zsxwing Jul 6, 2017
67cfdb4
[SPARK-21308][SQL] Remove SQLConf parameters from the optimizer
gatorsmile Jul 6, 2017
1c1ef68
[SPARK-21312][SQL] correct offsetInBytes in UnsafeRow.writeToStream
Jul 6, 2017
ff1ba2f
[SS][MINOR] Fix flaky test in DatastreamReaderWriterSuite. temp check…
tdas Jul 6, 2017
95ef2d3
[SPARK-21012][SUBMIT] Add glob support for resources adding to Spark
jerryshao Jul 6, 2017
c3c4ba7
[SPARK-20703][SQL] Associate metrics with data writes onto DataFrameW…
viirya Jul 6, 2017
5f6089f
[SPARK-21324][TEST] Improve statistics test suites
wzhfy Jul 6, 2017
41ca6e2
[SPARK-21273][SQL][FOLLOW-UP] Add missing test cases back and revise …
gengliangwang Jul 6, 2017
b3bb076
[SPARK-20950][CORE] add a new config to diskWriteBufferSize which is …
heary-cao Jul 6, 2017
3806e01
[SPARK-21228][SQL] InSet incorrect handling of structs
bogdanrdc Jul 6, 2017
96ff87d
[SPARK-21204][SQL] Add support for Scala Set collection types in seri…
viirya Jul 6, 2017
d233754
[SPARK-21323][SQL] Rename plans.logical.statsEstimation.Range to Valu…
gengliangwang Jul 6, 2017
e0e6005
[SPARK-21267][SS][DOCS] Update Structured Streaming Documentation
tdas Jul 7, 2017
97a6fa5
[SPARK-20946][SQL] Do not update conf for existing SparkContext in Sp…
cloud-fan Jul 7, 2017
716fba9
[SPARK-21329][SS] Make EventTimeWatermarkExec explicitly UnaryExecNode
jaceklaskowski Jul 7, 2017
e9137db
[SPARK-21326][SPARK-21066][ML] Use TextFileFormat in LibSVMFileFormat…
HyukjinKwon Jul 7, 2017
45fcecc
[SPARK-21327][SQL][PYSPARK] ArrayConstructor should handle an array o…
ueshin Jul 7, 2017
d4cd503
[SPARK-21217][SQL] Support ColumnVector.Array.to<type>Array()
kiszk Jul 7, 2017
b4d1299
[SPARK-20703][SQL][FOLLOW-UP] Associate metrics with data writes onto…
viirya Jul 7, 2017
c53a184
[SPARK-21313][SS] ConsoleSink's string representation
jaceklaskowski Jul 7, 2017
0c26cea
[SPARK-21285][ML] VectorAssembler reports the column name of unsuppor…
facaiy Jul 7, 2017
eaa24cb
[SPARK-21335][SQL] support un-aliased subquery
cloud-fan Jul 7, 2017
8a3119a
[SPARK-19358][CORE] LiveListenerBus shall log the event name when dro…
CodingCat Jul 7, 2017
19e1861
[SPARK-21336] Revise rand comparison in BatchEvalPythonExecSuite
gengliangwang Jul 7, 2017
000bcd4
[SPARK-21100][SQL] Add summary method as alternative to describe that…
aray Jul 8, 2017
e55a618
[SPARK-21281][SQL] Use string types by default if array and map have …
maropu Jul 8, 2017
443d8ec
[SPARK-20379][CORE] Allow SSL config to reference env variables.
Jul 8, 2017
2a24425
[SPARK-21069][SS][DOCS] Add rate source to programming guide.
ScrapCodes Jul 8, 2017
f151681
[SPARK-20307][SPARKR] SparkR: pass on setHandleInvalid to spark.mllib…
wangmiao1981 Jul 8, 2017
412bf55
[SPARK-20456][DOCS] Add examples for functions collection for pyspark
map222 Jul 8, 2017
0f98c0e
Mesos doc fixes
Jul 8, 2017
243d007
[SPARK-20609][MLLIB][TEST] manually cleared 'spark.local.dir' before/…
heary-cao Jul 8, 2017
f2bcffb
[SPARK-21345][SQL][TEST][TEST-MAVEN] SparkSessionBuilderSuite should …
dongjoon-hyun Jul 8, 2017
e526698
[SPARK-21083][SQL] Store zero size and row count when analyzing empty…
wzhfy Jul 8, 2017
fd89fe4
[SPARK-20342][CORE] Update task accumulators before sending task end …
Jul 8, 2017
8df71fc
[SPARK-21343] Refine the document for spark.reducer.maxReqSizeShuffle…
Jul 8, 2017
40b1b9f
[SPARK-21307][REVERT][SQL] Remove SQLConf parameters from the parser-…
gatorsmile Jul 8, 2017
a0b3f89
[SPARK-21093][R] Terminate R's worker processes in the parent of R's …
HyukjinKwon Jul 8, 2017
b8f49ff
[SPARK-18016][SQL][FOLLOWUP] merge declareAddedFunctions, initNestedC…
cloud-fan Jul 9, 2017
bd6e6de
[MINOR][DOC] Improve the docs about how to correctly set configurations
jerryshao Jul 10, 2017
7132f28
[SPARK-21100][SQL][FOLLOWUP] cleanup code and add more comments for D…
cloud-fan Jul 10, 2017
e81cebc
[SPARK-21219][CORE] Task retry occurs on same executor due to race co…
Jul 10, 2017
160cf66
[MINOR][DOC] Remove obsolete `ec2-scripts.md`
dongjoon-hyun Jul 10, 2017
52bf61f
[SPARK-20460][SQL] Make it more consistent to handle column name dupl…
maropu Jul 10, 2017
61d61b7
[SPARK-21342] Fix DownloadCallback to work well with RetryingBlockFet…
Jul 10, 2017
4aa593e
[SPARK-21272] SortMergeJoin LeftAnti does not update numOutputRows
juliuszsompolski Jul 10, 2017
c2c5a72
[SPARK-21266][R][PYTHON] Support schema a DDL-formatted string in dap…
HyukjinKwon Jul 10, 2017
f770d7b
[SPARK-13534][PYSPARK] Using Apache Arrow to increase performance of …
BryanCutler Jul 10, 2017
a5b514c
[SPARK-21358][EXAMPLES] Argument of repartitionandsortwithinpartition…
chie8842 Jul 11, 2017
138914a
[SPARK-21043][SQL] Add unionByName in Dataset
maropu Jul 11, 2017
a1dd388
[SPARK-21350][SQL] Fix the error message when the number of arguments…
gatorsmile Jul 11, 2017
b883b4e
[SPARK-21369][CORE] Don't use Scala Tuple2 in common/network-*
zsxwing Jul 11, 2017
4ba91e2
[SPARK-21315][SQL] Skip some spill files when generateIterator(startI…
Jul 11, 2017
3576942
[SPARK-20456][PYTHON][FOLLOWUP] Fix timezone-dependent doctests in un…
HyukjinKwon Jul 11, 2017
5c8db23
[SPARK-20331][SQL] Enhanced Hive partition pruning predicate pushdown
Jul 11, 2017
8435d14
[SPARK-21263][SQL] Do not allow partially parsing double and floats v…
HyukjinKwon Jul 11, 2017
8f01e36
[SPARK-21366][SQL][TEST] Add sql test for window functions
jiangxb1987 Jul 11, 2017
46083ec
[SPARK-21365][PYTHON] Deduplicate logics parsing DDL type/schema defi…
HyukjinKwon Jul 11, 2017
cb22463
[SPARK-16019][YARN] Use separate RM poll interval when starting clien…
Jul 11, 2017
45853d5
[SPARK-19285][SQL] Implement UDF0
gatorsmile Jul 11, 2017
64c610e
[SPARK-12139][SQL] REGEX Column Specification
janewangfb Jul 12, 2017
e836c3b
[SPARK-21382] The note about Scala 2.10 in building-spark.md is wrong.
liu-zhaokun Jul 12, 2017
9758401
[SPARK-21146][CORE] Master/Worker should handle and shutdown when any…
Jul 12, 2017
aa9dd02
[SPARK-21370][SS] Add test for state reliability when one read-only s…
brkyvz Jul 12, 2017
6c0f551
[SPARK-20842][SQL] Upgrade to 1.2.2 for Hive Metastore Client 1.2
gatorsmile Jul 12, 2017
785ef8b
[SPARK-21305][ML][MLLIB] Add options to disable multi-threading of na…
Jul 12, 2017
17d774b
[SPARK-21007][SQL] Add SQL function - RIGHT && LEFT
10110346 Jul 12, 2017
f0eaa37
[SPARK-18619][ML] Make QuantileDiscretizer/Bucketizer/StringIndexer/R…
zhengruifeng Jul 12, 2017
8eaf389
[SPARK-17701][SQL] Refactor RowDataSourceScanExec so its sameResult c…
cloud-fan Jul 12, 2017
a6ea634
[SPARK-18646][REPL] Set parent classloader as null for ExecutorClassL…
taroplus Jul 13, 2017
90a083c
[SPARK-19810][BUILD][CORE] Remove support for Scala 2.10
srowen Jul 13, 2017
633307f
[SPARK-21373][CORE] Update Jetty to 9.3.20.v20170531
kiszk Jul 13, 2017
4ffbfc9
[SPARK-21403][MESOS] fix --packages for mesos
skonto Jul 13, 2017
24db217
[SPARK-15526][MLLIB] Shade JPMML
srowen Jul 13, 2017
d07b62b
[SPARK-21376][YARN] Fix yarn client token expire issue when cleaning …
jerryshao Jul 13, 2017
153ba77
[SPARK-9825][YARN] Do not overwrite final Hadoop config entries.
Jul 14, 2017
1b7f784
[SPARK-21421][SS] Add the query id as a local property to allow sourc…
zsxwing Jul 14, 2017
a974595
[SPARK-21344][SQL] BinaryType comparison does signed byte array compa…
kiszk Jul 15, 2017
f9c8817
[SPARK-21267][DOCS][MINOR] Follow up to avoid referencing programming…
srowen Jul 15, 2017
e844d77
[SPARK-20307][ML][SPARKR][FOLLOW-UP] RFormula should handle invalid f…
yanboliang Jul 15, 2017
905d28e
[SPARK-19810][SPARK-19810][MINOR][FOLLOW-UP] Follow-ups from to remov…
srowen Jul 17, 2017
baf83f1
[SPARK-21354][SQL] INPUT FILE related functions do not support more t…
gatorsmile Jul 17, 2017
0fdff9f
[SPARK-21394][SPARK-21432][PYTHON] Reviving callable object/partial f…
HyukjinKwon Jul 17, 2017
728843f
[SPARK-21221][ML] CrossValidator and TrainValidationSplit Persist Nes…
ajaysaini725 Jul 17, 2017
e5d3318
[SPARK-21321][SPARK CORE] Spark very verbose on shutdown
Jul 17, 2017
566927a
[SPARK-21377][YARN] Make jars specify with --jars/--packages load-abl…
jerryshao Jul 17, 2017
d9cfac4
[SPARK-21409][SS] Expose state store memory usage in SQL metrics and …
tdas Jul 17, 2017
009b4dc
[MINOR] Improve SQLConf messages
gatorsmile Jul 18, 2017
e9bd004
[SPARK-21410][CORE] Create less partitions for RangePartitioner if RD…
apapi Jul 18, 2017
e801d8b
[SPARK-21409][SS] Follow up PR to allow different types of custom met…
tdas Jul 18, 2017
7f9fbc6
[SPARK-21444] Be more defensive when removing broadcasts in MapOutput…
JoshRosen Jul 18, 2017
2d93182
[SPARK-21332][SQL] Incorrect result type inferred for some decimal ex…
Jul 18, 2017
bdeb44a
[SPARK-21445] Make IntWrapper and LongWrapper in UTF8String Serializable
brkyvz Jul 18, 2017
0c42ab0
[SPARK-21415] Triage scapegoat warnings, part 1
srowen Jul 18, 2017
e2d70ee
[SPARK-15526][ML][FOLLOWUP] Make JPMML provided scope to avoid includ…
srowen Jul 18, 2017
3c4f7f3
[SPARK-21411][YARN] Lazily create FS within kerberized UGI to avoid t…
jerryshao Jul 18, 2017
13c38ac
[SPARK-21408][CORE] Better default number of RPC dispatch threads.
Jul 18, 2017
b56ea7d
[SPARK-21457][SQL] ExternalCatalog.listPartitions should correctly ha…
cloud-fan Jul 18, 2017
136f414
[SPARK-21462][SS] Added batchId to StreamingQueryProgress.json
tdas Jul 18, 2017
5e233ee
[SPARK-21435][SQL] Empty files should be skipped while write to file
xuanyuanking Jul 19, 2017
f738b18
[SPARK-21273][SQL][FOLLOW-UP] Propagate logical plan stats using visi…
gatorsmile Jul 19, 2017
0643b3e
[SPARK-21401][ML][MLLIB] add poll function for BoundedPriorityQueue
Jul 19, 2017
16d0036
[SPARK-21414] Refine SlidingWindowFunctionFrame to avoid OOM.
Jul 19, 2017
939a213
[SPARK-21441][SQL] Incorrect Codegen in SortMergeJoinExec results fai…
DonnyZone Jul 19, 2017
327ba66
[SPARK-21464][SS] Minimize deprecation warnings caused by ProcessingT…
tdas Jul 19, 2017
67a4561
[SPARK-21243][Core] Limit no. of map outputs in a shuffle fetch
dhruve Jul 19, 2017
ddfdeec
[SPARK-21446][SQL] Fix setAutoCommit never executed
DFFuture Jul 19, 2017
0d8a3ad
[SPARK-21456][MESOS] Make the driver failover_timeout configurable
susanxhuynh Jul 19, 2017
97391d9
[SPARK-21333][DOCS] Removed invalid joinTypes from javadoc of Dataset…
coreywoodfield Jul 19, 2017
74e06b9
[SPARK-21463] Allow userSpecifiedSchema to override partition inferen…
brkyvz Jul 19, 2017
1010d28
[SPARK-16542][SQL][PYSPARK] Fix bugs about types that result an array…
zasdfgbnm Jul 20, 2017
276f9b6
[MINOR][DOCS] Fix some missing notes for Python 2.6 support drop
HyukjinKwon Jul 20, 2017
cf656c1
[SPARK-21477][SQL][MINOR] Mark LocalTableScanExec's input data transient
gatorsmile Jul 20, 2017
7140b14
[MINOR][ML] Reorg RFormula params.
yanboliang Jul 20, 2017
3c12899
[SPARK-21472][SQL] Introduce ArrowColumnVector as a reader for Arrow …
ueshin Jul 20, 2017
60f95ed
[SPARK-19531] Send UPDATE_LENGTH for Spark History service
dosoft Jul 20, 2017
acdb3f6
[SPARK-21142][SS] spark-streaming-kafka-0-10 should depend on kafka-c…
timvw Jul 20, 2017
adbbf72
[SPARK-10063] Follow-up: remove dead code related to an old output co…
cloud-fan Jul 20, 2017
2476c84
[MINOR][SS][DOCS] Minor doc change for kafka integration
viirya Jul 21, 2017
d899699
[SPARK-21472][SQL][FOLLOW-UP] Introduce ArrowColumnVector as a reader…
ueshin Jul 21, 2017
651887b
[SPARK-19810][BUILD][FOLLOW-UP] jcl-over-slf4j dependency needs to be…
srowen Jul 21, 2017
69226f2
[SPARK-21434][PYTHON][DOCS] Add pyspark pip documentation.
holdenk Jul 21, 2017
ed4171c
[SPARK-10063] Follow-up: remove a useless test related to an old outp…
cloud-fan Jul 23, 2017
c56c0a1
[SPARK-20904][CORE] Don't report task failures to driver during shutd…
Jul 23, 2017
51fe678
[SPARK-20871][SQL] limit logging of Janino code
Jul 23, 2017
4862ac9
[MINOR] Remove **** in test case names in FlatMapGroupsWithStateSuite
rxin Jul 23, 2017
f8e2539
[SPARK-21512][SQL][TEST] DatasetCacheSuite needs to execute unpersist…
kiszk Jul 23, 2017
d97ea8d
[SPARK-17528][SQL][FOLLOWUP] remove unnecessary data copy in object h…
cloud-fan Jul 24, 2017
bb101a0
[SPARK-21502][MESOS] fix --supervise for mesos in cluster mode
skonto Jul 24, 2017
b2e29ad
[SPARK-21516][SQL][TEST] Overriding afterEach() in DatasetCacheSuite …
kiszk Jul 25, 2017
44db5f2
[SPARK-20855][Docs][DStream] Update the Spark kinesis docs to use the…
yashs360 Jul 25, 2017
5e24af6
[SPARK-21498][EXAMPLES] quick start -> one py demo have some bug in code
lizhaoch Jul 25, 2017
2e6e456
[SPARK-21175] Reject OpenBlocks when memory shortage on shuffle service.
Jul 25, 2017
57767de
[SPARK-21383][YARN] Fix the YarnAllocator allocates more Resource
Jul 25, 2017
8bb3d5b
[SPARK-21447][WEB UI] Spark history server fails to render compressed
Jul 25, 2017
a4986e3
[SPARK-21491][GRAPHX] Enhance GraphX performance: breakOut instead of…
SereneAnt Jul 25, 2017
c98e5b2
[SPARK-20586][SQL] Add deterministic to ScalaUDF
gatorsmile Jul 26, 2017
a508057
[SPARK-21494][NETWORK] Use correct app id when authenticating to exte…
Jul 26, 2017
88d592f
[SPARK-21517][CORE] Avoid copying memory when transfer chunks remotely
caneMi Jul 26, 2017
3422833
[SPARK-21524][ML] unit test fix: ValidatorParamsSuiteHelpers generate…
YY-OnCall Jul 26, 2017
fe421e4
[SPARK-20988][ML] Logistic regression uses aggregator hierarchy
sethah Jul 26, 2017
30153ce
[SPARK-21485][SQL][DOCS] Spark SQL documentation generation for built…
HyukjinKwon Jul 26, 2017
37956d6
[SPARK-21530] Update description of spark.shuffle.maxChunksBeingTrans…
Jul 27, 2017
7548a82
[SPARK-21271][SQL] Ensure Unsafe.sizeInBytes is a multiple of 8
kiszk Jul 27, 2017
1a331fc
[SPARK-21440][SQL][PYSPARK] Refactor ArrowConverters and add ArrayTyp…
ueshin Jul 27, 2017
76f7eb3
[SPARK-19270][ML] Add summary table to GLM summary
actuaryzhang Jul 27, 2017
773fda7
[SPARK-21319][SQL] Fix memory leak in sorter
cloud-fan Jul 27, 2017
98a9872
[SPARK-21538][SQL] Attribute resolution inconsistency in the Dataset API
Jul 27, 2017
b72bc2e
[SPARK-21306][ML] OneVsRest should support setWeightCol
facaiy Jul 28, 2017
7d10aa4
[MINOR][BUILD] Fix current lint-java failures
srowen Jul 28, 2017
750296e
[SPARK-21553][SPARK SHELL] Add the description of the default value o…
Jul 28, 2017
b1c515f
[SPARK-21541][YARN] Spark Logs show incorrect job status for a job th…
Jul 28, 2017
fcc2090
Typo in comment
nahoj Jul 28, 2017
81c9625
[SPARK-20090][PYTHON] Add StructType.fieldNames in PySpark
HyukjinKwon Jul 29, 2017
530d59d
[SPARK-21508][DOC] Fix example code provided in Spark Streaming Docum…
Jul 29, 2017
a021193
[SPARK-21357][DSTREAMS] FileInputDStream not remove out of date RDD
shaofei007 Jul 29, 2017
f539d3c
[SPARK-21555][SQL] RuntimeReplaceable should be compared semantically…
viirya Jul 29, 2017
d17a82f
[SPARK-19451][SQL] rangeBetween method should accept Long value as bo…
jiangxb1987 Jul 29, 2017
141fdfe
[SPARK-20962][SQL] Support subquery column aliases in FROM clause
maropu Jul 29, 2017
93f8503
[SQL] Fix typo in DataframeWriter doc
Jul 30, 2017
72afc6f
[SPARK-21297][WEB-UI] Add count in 'JDBC/ODBC Server' page.
Jul 30, 2017
793818d
[MINOR][DOC] Replace numTasks with numPartitions in programming guide
polarker Jul 30, 2017
a38e99b
[MINOR] Minor comment fixes in merge_spark_pr.py script
HyukjinKwon Jul 31, 2017
11d8139
[SPARK-19839][CORE] release longArray in BytesToBytesMap
Jul 31, 2017
25dd91a
[SPARK-21575][SPARKR] Eliminate needless synchronization in java-R se…
SereneAnt Jul 31, 2017
8e4ee38
[SPARK-18950][SQL] Report conflicting fields when merging two StructT…
jiayue-zhang Aug 1, 2017
610c1c3
[SPARK-21381][SPARKR] SparkR: pass on setHandleInvalid for classifica…
wangmiao1981 Aug 1, 2017
2cbc234
[SPARK-21589][SQL][DOC] Add documents about Hive UDF/UDTF/UDAF
maropu Aug 1, 2017
5c3b4cd
[SPARK-21475][CORE] Use NIO's Files API to replace FileInputStream/Fi…
jerryshao Aug 1, 2017
37840a7
[SPARK-21388][ML][PYSPARK] GBTs inherit from HasStepSize & LInearSVC …
zhengruifeng Aug 1, 2017
157dbc3
[SPARK-21585] Application Master marking application status as Failed…
Aug 1, 2017
59af350
[SPARK-21522][CORE] Fix flakiness in LauncherServerSuite.
Aug 1, 2017
4ac3cf3
[SPARK-20079][YARN] Fix client AM not allocating executors after rest…
Aug 1, 2017
4980f4d
[SPARK-21592][BUILD] Skip maven-compiler-plugin main and test compila…
gslowikowski Aug 1, 2017
17471a2
[SPARK-21593][DOCS] Fix 2 rendering errors on configuration page
srowen Aug 1, 2017
58e5971
[SPARK-21339][CORE] spark-shell --packages option does not add jars t…
Aug 1, 2017
aa805e9
[SPARK-12717][PYTHON] Adding thread-safe broadcast pickle registry
BryanCutler Aug 1, 2017
86bf812
[CORE][MINOR] Improve the error message of checkpoint RDD verification
gatorsmile Aug 2, 2017
08b20e4
[SPARK-21578][CORE] Add JavaSparkContextSuite
dongjoon-hyun Aug 2, 2017
0413ab8
[SPARK-20601][ML] Python API for Constrained Logistic Regression
zero323 Aug 2, 2017
4c22de6
[SPARK-21597][SS] Fix a potential overflow issue in EventTimeStats
zsxwing Aug 2, 2017
1befdeb
[SPARK-21490][CORE] Make sure SparkLauncher redirects needed streams.
Aug 2, 2017
c480683
[SPARK-21546][SS] dropDuplicates should ignore watermark when it's no…
zsxwing Aug 2, 2017
ab8723f
[SPARK-21615][ML][MLLIB][DOCS] Fix broken redirect in collaborative f…
Aug 3, 2017
ffab059
[SPARK-21604][SQL] if the object extends Logging, i suggest to remove…
Aug 3, 2017
5cd3f5b
[SPARK-21611][SQL] Error class name for log in several classes.
Aug 3, 2017
b8bc9e4
[SPARK-21605][BUILD] Let IntelliJ IDEA correctly detect Language leve…
baibaichen Aug 3, 2017
54e084a
[SPARK-21602][R] Add map_keys and map_values functions to R
HyukjinKwon Aug 3, 2017
d009fb4
[SPARK-21599][SQL] Collecting column statistics for datasource tables…
dilipbiswal Aug 3, 2017
9de8c26
[SPARK-20713][SPARK CORE] Convert CommitDenied to TaskKilled.
Aug 3, 2017
ca8c7f9
improved test, and tested
Aug 4, 2017
04681bc
remove extra docs
Aug 4, 2017
969d049
style
Aug 4, 2017
77d2791
wip, added support for multiple secrets, reference-based and value-based
Aug 8, 2017
cd6ffd5
env-based secrets, fixed up tests, and cleaned up code
Aug 8, 2017
b9ff768
only allow one secret type at a time
Aug 8, 2017
c55ab81
move driverlabels out
Aug 8, 2017
8bd8e42
update running-on-mesos docs
Aug 9, 2017
a09d57a
addressed comments
Aug 11, 2017
70ede47
rebase
Aug 24, 2017
ba119fe
updated deps
Aug 24, 2017
e28f201
merge master
Aug 24, 2017
021ead1
addressed comments
Aug 25, 2017
f235677
changed config keys, some nits
Aug 29, 2017
60e50f1
change config keys in tests too
Aug 29, 2017
0d7674b
finished addressing comments, updated docs to match
Aug 29, 2017
2e75af7
a little more cleanup
Aug 29, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
move driverlabels out
  • Loading branch information
ArtRand committed Aug 24, 2017
commit c55ab812833582994d10c1b1d05d1126d7e75b5b
Original file line number Diff line number Diff line change
Expand Up @@ -568,15 +568,19 @@ private[spark] class MesosClusterScheduler(
offer.remainingResources = finalResources.asJava

val appName = desc.conf.get("spark.app.name")
val taskInfo = TaskInfo.newBuilder()

val driverLabels = MesosProtoUtils.mesosLabels(desc.conf.get(config.DRIVER_LABELS)
.getOrElse(""))

TaskInfo.newBuilder()
.setTaskId(taskId)
.setName(s"Driver for ${appName}")
.setSlaveId(offer.offer.getSlaveId)
.setCommand(buildDriverCommand(desc))
.setContainer(getContainerInfo(desc))
.addAllResources(cpuResourcesToUse.asJava)
.addAllResources(memResourcesToUse.asJava)
.setLabels(MesosProtoUtils.mesosLabels(desc.conf.get(config.DRIVER_LABELS).getOrElse("")))
.setLabels(driverLabels)
.build
}

Expand Down