Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Site: Troubleshooting/Website publishing improvements in release guide #3517

Closed
wants to merge 2 commits into from
Closed

Site: Troubleshooting/Website publishing improvements in release guide #3517

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7188517
Site: Troubleshooting/Website publishing improvements in release guide
libenchao Nov 11, 2023
79a38f0
Add more comments about asfGitSourceUsername/asfGitSourcePassword
libenchao Nov 14, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 20 additions & 11 deletions site/_docs/howto.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -675,17 +675,19 @@ asfGitSourceUsername=
asfGitSourcePassword=
{% endhighlight %}

Note: Both `asfNexusUsername` and `asfSvnUsername` are your apache id with `asfNexusPassword` and
Note:
* Both `asfNexusUsername` and `asfSvnUsername` are your apache id with `asfNexusPassword` and
`asfSvnPassword` are corresponding password.
* Git source account can be configured to either Gitbox (the default) or Github. For Gitbox, `asfGitSourceUsername`
is your apache id, and `asfGitSourcePassword` is the corresponding password. For Github, `asfGitSourceUsername`
is your GitHub id while `asfGitSourcePassword` is not your GitHub password, you need to generate it in
https://github.com/settings/tokens choosing `Personal access tokens`.

When
[asflike-release-environment](https://github.com/vlsi/asflike-release-environment)
is used, the credentials are taken from
`asfTest...` (e.g. `asfTestNexusUsername=test`)

Note: `asfGitSourceUsername` is your GitHub id while `asfGitSourcePassword` is not your GitHub password.
You need to generate it in https://github.com/settings/tokens choosing `Personal access tokens`.

Note: if you want to use `gpg-agent`, you need to pass some more properties:

{% highlight properties %}
Expand Down Expand Up @@ -806,6 +808,7 @@ git clean -xn
./gradlew prepareVote -Prc=0

# Push release candidate to ASF servers
# If you prefer to use Github account, change pushRepositoryProvider to GITHUB
./gradlew prepareVote -Prc=0 -Pasf -Pasf.git.pushRepositoryProvider=GITBOX
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Throws following exception with GITBOX:

> Task :pushRcTag FAILED

Build calcite FAILURE reason:
    Execution failed for task ':pushRcTag':
        Caused by: org.eclipse.jgit.api.errors.TransportException: https://gitbox.apache.org/repos/asf/calcite.git: not authorized
            at org.eclipse.jgit.api.PushCommand.call(PushCommand.java:147)
            at com.github.vlsi.gradle.release.jgit.dsl.GitExtensionsKt.push(GitExtensions.kt:132)
            at com.github.vlsi.gradle.release.GitPushTask$pushTag$1.invoke(GitPushTask.kt:54)
            at com.github.vlsi.gradle.release.GitPushTask$pushTag$1.invoke(GitPushTask.kt:30)
            at com.github.vlsi.gradle.release.DefaultGitTask.jgit(DefaultGitTask.kt:45)
            at com.github.vlsi.gradle.release.GitPushTask.pushTag(GitPushTask.kt:50)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For avatica, I also have -Pasf.git.pushRepositoryProvider=GITBOX in the release script: https://github.com/apache/calcite-avatica/blob/main/docker.sh#L298

I haven't used this recently, so not sure if it still works.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CC @NobiGo @zabetak since you are the latest release managers.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The asf.git.pushRepositoryProvider should be inline with asfGitSourceUsername and asfGitSourcePassword. If you are using GITBOX then you should use the ASF credentials; if you use GITHUB you should use the GitHub credentials. I assume both providers should work fine if the correct credentials are set.

I think there were some misconception that GITHUB was not working in the past (https://issues.apache.org/jira/browse/CALCITE-4856) but I suspect that the underlying reason was the same (mixing ASF vs. GitHub credentials).

If we want to improve something it makes sense to clarify somewhere this story about the different credentials and not just change again the default values.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was following this line to setup the environment:

calcite/site/_docs/howto.md

Lines 686 to 687 in 816edd1

Note: `asfGitSourceUsername` is your GitHub id while `asfGitSourcePassword` is not your GitHub password.
You need to generate it in https://github.com/settings/tokens choosing `Personal access tokens`.

I thought this is the suggested way to interact with Git. (Actually I'm not sure how to setup Gitbox credentials, I guess asfGitSourcePassword is the apache password which is the same with asfNexusPassword and
asfSvnPassword?)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the Gitbox credentials is the usual username and password combination that is used to access most (if not all) ASF services.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll leave the default as 'GITBOX', and add more comments about it.

{% endhighlight %}

Expand Down Expand Up @@ -927,12 +930,6 @@ thread to discuss.
Julian
{% endhighlight %}

Use the [Apache URL shortener](https://s.apache.org) to generate
shortened URLs for the vote proposal and result emails. Examples:
[s.apache.org/calcite-1.2-vote](https://s.apache.org/calcite-1.2-vote) and
[s.apache.org/calcite-1.2-result](https://s.apache.org/calcite-1.2-result).
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems there are no steps need to use this, I propose to remove it to make the document clean.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, Since the release script generates the emails, I think most RMs will just copy that verbatim, so the shortening of links seem redundant.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding is that these steps are for creating mnemonic URLs that people can easily find and refer to it in future. I don't think was ever meant to be used for the voting process it self but for later reference to it if needed. I don't have a strong opinion on keeping or removing them but they do serve a purpose. For instance, typing the following just works and brings me what I need:

Copy link
Member Author

@libenchao libenchao Nov 13, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also agree that it served for some purpose. The problem now is that it sometimes confuses new RMs (this has happened during 1.35.0 releasing: https://lists.apache.org/thread/nv2xclvgrw84866o0q2ngdx4c2dg6pqq).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the explanation that I gave above clarifies a bit the situation then we could enrich the doc accordingly. If you feel that there might be still confusion then feel free to remove the section altogether.



## Publishing a release

After a successful release vote, we need to push the release
Expand All @@ -948,6 +945,7 @@ Remember that UTC date changes at 4 pm Pacific time.
./gradlew publishDist -Prc=0

# Publish the release to ASF servers
# If you prefer to use Github account, change pushRepositoryProvider to GITHUB
./gradlew publishDist -Prc=0 -Pasf -Pasf.git.pushRepositoryProvider=GITBOX
{% endhighlight %}

Expand All @@ -956,6 +954,15 @@ If for whatever reason the `publishDist` task fails
it is still possible to perform the publishing tasks manually. Ask for help in the dev list if
you are not sure what needs to be done.

If the `releaseRepository` task prints something like:
{% highlight text%}
> Task :releaseRepository
Initialized stagingRepositoryId orgapachecalcite-1219 for repository nexus
GET request failed. 404: Not Found, body: [errors:[[id:*, msg:No such repository: orgapachecalcite-1219]]]
Requested operation was executed successfully in attempt 83 (maximum allowed 601)
{% endhighlight %}
it's most likely that the repository has been successfully released, you can check it in [ASF Nexus](https://repository.apache.org/).

Svnpubsub will publish to the
[release repo](https://dist.apache.org/repos/dist/release/calcite) and propagate to the
[mirrors](https://www.apache.org/dyn/closer.cgi/calcite) almost immediately.
Expand Down Expand Up @@ -983,7 +990,9 @@ Add a release announcement by copying
[site/_posts/2016-10-12-release-1.10.0.md]({{ site.sourceRoot }}/site/_posts/2016-10-12-release-1.10.0.md),
and adapt the release date in `history.md` if necessary. Preview the changes locally, by following the
instructions in [site/README.md]({{ site.sourceRoot }}/site/README.md), and then commit and push
the changes to the `main` branch.
the changes to the `main` branch. Please note that due to [CALCITE-5584](https://issues.apache.org/jira/browse/CALCITE-5584),
the commit should be pushed to Github as the last commit, do not chain it with "Prepare for next development iteration"
commit.

Ensure that all changes to the website (news, release notes, javadoc) are correctly displayed.

Expand Down