-
Notifications
You must be signed in to change notification settings - Fork 28.6k
[SPARK-11631][Scheduler] Adding 'Starting DAGScheduler' log #9603
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Can one of the admins verify this patch? |
@@ -506,6 +506,7 @@ class SparkContext(config: SparkConf) extends Logging with ExecutorAllocationCli | |||
val (sched, ts) = SparkContext.createTaskScheduler(this, master) | |||
_schedulerBackend = sched | |||
_taskScheduler = ts | |||
logDebug("Starting DAGScheduler") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd rather move it to DAGScheduler's constructor as that's where the message belongs.
Meh, I don't really think this is worth doing. It doesn't add any value or fix anything. |
I agree it is trivial, just thought I could quickly add a log statement. If Jacek agrees, I can close those PR. |
I don't agree with @andrewor14. It does add a value of being consistent with how Spark informs about its status - if it says "Stopping..." at INFO it should be corresponding "Starting..." at INFO. That was my initial goal. Consistency is the goal and the value here. Why does Spark report about services being stopped at all? What's the rationale? I would change it to DEBUG, but since all the other logs about starting services are at INFO, I'd leave it as is until someone reports it should be done. |
@andrewor14 What is your take on Jacek's comment? I don't think it's a bad idea to make it more consistent with a matching log message. Please let me know. Thx! |
I just don't see any value. It's obvious that the scheduler is starting / stopping. Not a big deal if we merge it, but I just can't imagine anyone finding this message useful. |
Beyond that, the message is actually somewhat misleading. The "Stopping" message occurs in |
I think this message is complete useless. I'd just remove it rather than spending time bickering about the wording. |
OK, I will close it. Thanks! |
No description provided.