[2.4][SPARK-26048][SPARK-24530] Cherrypick all the missing commits to 2.4 release script #24503
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…of arguments ## What changes were proposed in this pull request? The call to `translate_component` only supplied 2 out of the 3 required arguments. I added a default empty list for the missing argument to avoid a run-time error. I work for Semmle, and noticed the bug with our LGTM code analyzer: https://lgtm.com/projects/g/apache/spark/snapshot/0655f1624ff7b73e5c8937ab9e83453a5a3a4466/files/dev/create-release/releaseutils.py?sort=name&dir=ASC&mode=heatmap#x1434915b6576fb40:1 ## How was this patch tested? I checked that `./dev/run-tests` pass OK. Closes apache#23567 from ipwright/wrong-number-of-arguments-fix. Authored-by: wright <wright@semmle.com> Signed-off-by: Sean Owen <sean.owen@databricks.com>
## What changes were proposed in this pull request? SPARK-24530 discovered a problem of generation python doc, and provided a fix: setting SPHINXPYTHON to python 3. This PR makes this fix automatic in the release script using docker. ## How was this patch tested? verified by the 2.4.0 rc2 Closes apache#22607 from cloud-fan/python. Authored-by: Wenchen Fan <wenchen@databricks.com> Signed-off-by: Marcelo Vanzin <vanzin@cloudera.com>
## What changes were proposed in this pull request? When running the release script, you will be interrupted unexpectedly ``` ATTENTION! Your password for authentication realm: <https://dist.apache.org:443> ASF Committers can only be stored to disk unencrypted! You are advised to configure your system so that Subversion can store passwords encrypted, if possible. See the documentation for details. You can avoid future appearances of this warning by setting the value of the 'store-plaintext-passwords' option to either 'yes' or 'no' in '/home/spark-rm/.subversion/servers'. ----------------------------------------------------------------------- Store password unencrypted (yes/no)? ``` We can avoid it by adding `--no-auth-cache` when running svn command. ## How was this patch tested? manually verified with 2.4.0 RC5 Closes apache#22885 from cloud-fan/svn. Authored-by: Wenchen Fan <wenchen@databricks.com> Signed-off-by: Wenchen Fan <wenchen@databricks.com>
|
+1 for this approach because |
dongjoon-hyun
approved these changes
May 1, 2019
There was a problem hiding this comment.
+1, LGTM. Thank you so much, @gatorsmile .
This is a correct merged patch for four previous patches. Especially, for SPARK-26048, this is the correct fix. I hope there is all we need for branch-2.4. If something happens during RC building, please add more, @gatorsmile .
|
Test build #105053 has finished for PR 24503 at commit
|
srowen
approved these changes
May 1, 2019
gatorsmile
added a commit
that referenced
this pull request
May 1, 2019
… 2.4 release script ## What changes were proposed in this pull request? This PR is to cherry-pick all the missing and relevant commits that were merged to master but not to branch-2.4. Previously, dbtsai used the release script in the branch 2.4 to release 2.4.1. After more investigation, I found it is risky to make a 2.4 release by using the release script in the master branch since the release script has various changes. It could easily introduce unnoticeable issues, like what we did for 2.4.2. Thus, I would cherry-pick all the missing fixes and use the updated release script to release 2.4.3 ## How was this patch tested? N/A Closes #24503 from gatorsmile/upgradeReleaseScript. Lead-authored-by: Wenchen Fan <wenchen@databricks.com> Co-authored-by: gatorsmile <gatorsmile@gmail.com> Co-authored-by: wright <wright@semmle.com> Signed-off-by: gatorsmile <gatorsmile@gmail.com>
|
@dongjoon-hyun @srowen Thanks! Merged to 2.4 |
rluta
pushed a commit
to rluta/spark
that referenced
this pull request
Sep 17, 2019
… 2.4 release script ## What changes were proposed in this pull request? This PR is to cherry-pick all the missing and relevant commits that were merged to master but not to branch-2.4. Previously, dbtsai used the release script in the branch 2.4 to release 2.4.1. After more investigation, I found it is risky to make a 2.4 release by using the release script in the master branch since the release script has various changes. It could easily introduce unnoticeable issues, like what we did for 2.4.2. Thus, I would cherry-pick all the missing fixes and use the updated release script to release 2.4.3 ## How was this patch tested? N/A Closes apache#24503 from gatorsmile/upgradeReleaseScript. Lead-authored-by: Wenchen Fan <wenchen@databricks.com> Co-authored-by: gatorsmile <gatorsmile@gmail.com> Co-authored-by: wright <wright@semmle.com> Signed-off-by: gatorsmile <gatorsmile@gmail.com>
kai-chi
pushed a commit
to kai-chi/spark
that referenced
this pull request
Sep 26, 2019
… 2.4 release script ## What changes were proposed in this pull request? This PR is to cherry-pick all the missing and relevant commits that were merged to master but not to branch-2.4. Previously, dbtsai used the release script in the branch 2.4 to release 2.4.1. After more investigation, I found it is risky to make a 2.4 release by using the release script in the master branch since the release script has various changes. It could easily introduce unnoticeable issues, like what we did for 2.4.2. Thus, I would cherry-pick all the missing fixes and use the updated release script to release 2.4.3 ## How was this patch tested? N/A Closes apache#24503 from gatorsmile/upgradeReleaseScript. Lead-authored-by: Wenchen Fan <wenchen@databricks.com> Co-authored-by: gatorsmile <gatorsmile@gmail.com> Co-authored-by: wright <wright@semmle.com> Signed-off-by: gatorsmile <gatorsmile@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
This PR is to cherry-pick all the missing and relevant commits that were merged to master but not to branch-2.4.
Previously, @dbtsai used the release script in the branch 2.4 to release 2.4.1.
After more investigation, I found it is risky to make a 2.4 release by using the release script in the master branch since the release script has various changes. It could easily introduce unnoticeable issues, like what we did for 2.4.2.
Thus, I would cherry-pick all the missing fixes and use the updated release script to release 2.4.3
How was this patch tested?
N/A