SPARK-12105 - SPARK-SQL add convenient show functions #10130
Conversation
|
Rebase |
jbonofre
force-pushed
the
SPARK-12105
branch
5 times, most recently
from
b721fb2 to
61dd094
Compare
jbonofre
force-pushed
the
SPARK-12105
branch
from
61dd094 to
edbb87c
Compare
| writer.write(showString(numRows, truncate)) | ||
| } | ||
|
|
||
| def show(numRows: Int, truncate: Boolean, writer: Writer): Unit = { |
There was a problem hiding this comment.
these signatures are really clunky and I don't think we want it. Also the string is already composed so it's not like we're streaming things to the writer anyway.
|
@jbonofre I think it might be sufficient to just expose |
|
add to whitelist |
|
Test build #47589 has finished for PR 10130 at commit
|
|
Thanks for the update @andrewor14. It makes sense for the new functions(). Exposing the showString() is enough for most of the cases. Let me update the PR. |
jbonofre
force-pushed
the
SPARK-12105
branch
from
ff7dc62 to
219d75c
Compare
|
Test build #47594 has finished for PR 10130 at commit
|
| @@ -17,7 +17,7 @@ | |||
|
|
|||
| package org.apache.spark.sql | |||
|
|
|||
| import java.io.CharArrayWriter | |||
| import java.io.{OutputStreamWriter, Writer, OutputStream, CharArrayWriter} | |||
jbonofre
force-pushed
the
SPARK-12105
branch
from
219d75c to
694e94f
Compare
|
Test build #47720 has finished for PR 10130 at commit
|
| * Compose the string representing rows for output | ||
| */ | ||
| def showString(): String = { | ||
| showString(20); |
There was a problem hiding this comment.
remove ;, this is scala. I'll fix this on merge
|
Merged into master. |
|
Sorry I reverted the patch. Please check with me on the API change in the future. First I'm not 100% sure whether we want this (we might, just not sure yet), and then there are problems with the API (e.g. we should not use any default values for Java compatibility). Last but not least, if we want to do this, we should have it also for Python. |
|
Hi Reynold. No problem, I just proposed the PR as least to discuss, and I think it's convenient for users. Thanks. |
|
By the way, let me know if you want the change, I can extend to Python API as well. And what do you mean by default values about Java compatibility ? |
|
Default parameter values only work in Scala, not Java. |
|
Hi. So it's 2018 and I also vouch for having this method exposed as public to capture into logs in the driver program. Thanks! |
|
+1 any chance we can revive this PR ? |
|
+1 |
No description provided.