[chore][docs] add deploy spark/flink connectors to maven release repo…

… docs (apache#7616)
924060929 · Jan 6, 2022 · ad35067 · ad35067
1 parent 482bf05
commit ad35067
Show file tree

Hide file tree

Showing 10 changed files with 188 additions and 24 deletions.
diff --git a/docs/.vuepress/public/images/close_staging.png b/docs/.vuepress/public/images/close_staging.png
diff --git a/docs/.vuepress/public/images/release-stage.png b/docs/.vuepress/public/images/release-stage.png
diff --git a/docs/.vuepress/public/images/staging_repo.png b/docs/.vuepress/public/images/staging_repo.png
diff --git a/docs/en/community/how-to-deploy-to-maven.md b/docs/en/community/how-to-deploy-to-maven.md
@@ -75,7 +75,7 @@ under the License.
        <activeProfile>gpg</activeProfile>
      </activeProfiles>
    ```
-### Publish to SNAPSHOT
+## Publish to SNAPSHOT
 ### 1. Deploy flink connector
 
 Switch to the flink connector directory, let’s take flink version 1.11.6 and scalar 2.12 as examples
@@ -105,3 +105,87 @@ Switch to the spark connector directory, let’s take spark version 2.3.4 and sc
    export SCALA_VERSION="2.11"
    mvn deploy
    ```
+## Publish to Release
+
+### 1. Prepare GitHub authority
+Encrypt github token password: `mvn --encrypt-password <token>` This token is a token generated by github users to access the apache warehouse. The output cat is similar to `{bSSA+TC6wzEHNKukcOn...}` in `~/.m2/settings. Add to the `<servers>` tag in xml`
+```xml
+    <server>
+      <id>github</id>
+      <username>github user name</username>
+      <password>{bSSA+TC6wzEHNKukcOn...}</password>
+    </server>
+```
+
+### 2. Publish to  maven staging
+Take the release of Doris Flink Connector 1.0.0 as an example, the flink version is 1.13.5, the scala version is 2.12, and others are similar
+```bash
+cd extension/flink-doris-connector/
+export DORIS_HOME=$PWD/../../
+source ${DORIS_HOME}/env.sh
+export FLINK_VERSION=1.13.5
+export SCALA_VERSION=2.12
+mvn release:clean
+mvn release:prepare
+```
+After that, maven needs to enter three information
+1. The version information of Doris Flink Connector, we can do it by default, you can directly press Enter or enter the version you want
+2. The release tag of Doris Flink Connector, because the release process will add a tag in apache/incubator-doris, so a tag name is required. If it is default, just press Enter.
+3. The version number of the next version of Doris Flink Connector. Here we need to pay attention. Since our version number is {flink_version}_{scala_version}_1.0.0, maven's algorithm will increase the scala version number by one, if we want to use it next time This version of 1.0.1 can be changed to 1.13.5-2.12-1.0.1-SNAPSHOT
+```
+...
+[INFO] [prepare] 3/17 check-dependency-snapshots
+[INFO] Checking dependencies and plugins for snapshots ...
+[INFO] [prepare] 4/17 create-backup-poms
+[INFO] [prepare] 5/17 map-release-versions
+What is the release version for "Doris Flink Connector"? (org.apache.doris:doris-flink-connector) 1.13.5-2.12-1.0.0: :
+[INFO] [prepare] 6/17 input-variables
+What is the SCM release tag or label for "Doris Flink Connector"? (org.apache.doris:doris-flink-connector) doris-flink-connector-1.13.5-2.12-1.0.0: :
+[INFO] [prepare] 7/17 map-development-versions
+What is the new development version for "Doris Flink Connector"? (org.apache.doris:doris-flink-connector) 1.13.5-3.12-1.0.0-SNAPSHOT: : 1.13.5-2.12-1.0.1-SNAPSHOT
+...
+[INFO] [prepare] 17/17 end-release
+[INFO] Release preparation complete.
+[INFO] ------------------------------------------------------------------------
+[INFO] BUILD SUCCESS
+[INFO] ------------------------------------------------------------------------
+[INFO] Total time:  01:00 min
+[INFO] Finished at: 2022-01-05T15:01:55+08:00
+[INFO] ------------------------------------------------------------------------
+```
+Perform after prepare, and after perform, find the just released version in stagingRepositories in `https://repository.apache.org/`
+![](/images/staging_repo.png)
+
+### 3. vote
+You need to close staging before voting
+![](/images/close_staging.png)
+After that, you can vote in the dev@doris.apache.org, the following is an example of the mail
+```
+Hi,
+
+We are ready to deploy Doris connectors to Maven Central Repository, as the version 1.0.0, and the next version will be 1.0.1
+This Release contains Spark connectors and Flink connectors for spark 2/3 and flink 1.11/1.12/1.13
+
+GitHub tags:
+https://github.com/apache/incubator-doris/releases/tag/doris-spark-connector-3.1.2-2.12-1.0.0
+https://github.com/apache/incubator-doris/releases/tag/doris-spark-connector-2.3.4-2.11-1.0.0
+https://github.com/apache/incubator-doris/releases/tag/doris-flink-connector-1.13.5-2.12-1.0.0
+https://github.com/apache/incubator-doris/releases/tag/doris-flink-connector-1.12.7-2.12-1.0.0
+https://github.com/apache/incubator-doris/releases/tag/doris-flink-connector-1.11.6-2.12-1.0.0
+
+Staging repo:
+https://repository.apache.org/content/repositories/maven-[YOUR REPOSITORY ID]/
+https://repository.apache.org/content/repositories/maven-[YOUR REPOSITORY ID]/[PATH-TO]-source-release.zip
+
+Staging site:
+https://repository.apache.org/content/repositories/orgapachedoris-1000
+
+Vote open for at least 72 hours.
+
+[ ] +1
+[ ] +0
+[ ] -1
+```
+
+After the vote is passed, it can be published to Maven Central
+![](/images/release-stage.png)
diff --git a/docs/zh-CN/community/how-to-deploy-to-maven.md b/docs/zh-CN/community/how-to-deploy-to-maven.md
@@ -77,7 +77,7 @@ under the License.
        <activeProfile>gpg</activeProfile>
      </activeProfiles>
    ```
-### 发布到SNAPSHOT
+## 发布到SNAPSHOT
 ### 1. 部署 flink connector
 
 切换到 flink connector 目录， 我们以 flink 版本 1.11.6， scalar 2.12 为例
@@ -107,3 +107,85 @@ under the License.
    export SCALA_VERSION="2.11"
    mvn deploy
    ```
+
+## 发布到 Release
+
+### 1. 准备GitHub权限
+加密 github token 密码： `mvn --encrypt-password <token>` 这个token 是github 用户生成的访问apache仓库的token 输出cat类似`{bSSA+TC6wzEHNKukcOn...}` 在`~/.m2/settings.xml` 中的`<servers>` 标签中加入
+
+```xml
+    <server>
+      <id>github</id>
+      <username>github user name</username>
+      <password>{bSSA+TC6wzEHNKukcOn...}</password>
+    </server>
+```
+
+### 2. 发布到 maven staging
+以发布Doris Flink Connector 1.0.0 为例， flink 版本是 1.13.5 scala 版本是2.12, 其他类似
+```bash
+cd extension/flink-doris-connector/
+export DORIS_HOME=$PWD/../../
+source ${DORIS_HOME}/env.sh
+export FLINK_VERSION=1.13.5
+export SCALA_VERSION=2.12
+mvn release:clean
+mvn release:prepare
+```
+之后maven 需要输入三个信息 
+   1. Doris Flink Connector 的版本信息， 我们默认就可以，可以直接回车或者输入自己想要的版本
+   2. Doris Flink Connector 的release tag, 因为release 过程会在apache/incubator-doris中添加一个tag, 因此需要一个tag名称，如果默认，直接回车即可
+   3. Doris Flink Connector 下一个版本的版本号，这里需要注意，由于我们的版本号是 {flink_version}_{scala_version}_1.0.0 因此 maven 的算法会将scala 版本号加一，如果我们下次想用1.0.1 这个版本 我们可以改成1.13.5-2.12-1.0.1-SNAPSHOT
+```
+...
+[INFO] [prepare] 3/17 check-dependency-snapshots
+[INFO] Checking dependencies and plugins for snapshots ...
+[INFO] [prepare] 4/17 create-backup-poms
+[INFO] [prepare] 5/17 map-release-versions
+What is the release version for "Doris Flink Connector"? (org.apache.doris:doris-flink-connector) 1.13.5-2.12-1.0.0: :
+[INFO] [prepare] 6/17 input-variables
+What is the SCM release tag or label for "Doris Flink Connector"? (org.apache.doris:doris-flink-connector) doris-flink-connector-1.13.5-2.12-1.0.0: :
+[INFO] [prepare] 7/17 map-development-versions
+What is the new development version for "Doris Flink Connector"? (org.apache.doris:doris-flink-connector) 1.13.5-3.12-1.0.0-SNAPSHOT: : 1.13.5-2.12-1.0.1-SNAPSHOT
+...
+[INFO] [prepare] 17/17 end-release
+[INFO] Release preparation complete.
+[INFO] ------------------------------------------------------------------------
+[INFO] BUILD SUCCESS
+[INFO] ------------------------------------------------------------------------
+[INFO] Total time:  01:00 min
+[INFO] Finished at: 2022-01-05T15:01:55+08:00
+[INFO] ------------------------------------------------------------------------
+```
+prepare 之后 perform即可， perform 之后就在`https://repository.apache.org/` 里面的stagingRepositories 找到刚刚发布的版本
+![](/images/staging_repo.png)
+
+### 3. 投票
+投票前需要先close staging
+![](/images/close_staging.png)
+之后就可以在dev邮件组发起投票， 下面是一个邮件示例
+```
+Hi,
+
+We are ready to deploy Doris connectors to Maven Central Repository, as the version 1.0.0, and the next version will be 1.0.1
+This Release contains Spark connectors and Flink connectors for spark 2/3 and flink 1.11/1.12/1.13
+
+GitHub tags:
+https://github.com/apache/incubator-doris/releases/tag/doris-spark-connector-3.1.2-2.12-1.0.0
+https://github.com/apache/incubator-doris/releases/tag/doris-spark-connector-2.3.4-2.11-1.0.0
+https://github.com/apache/incubator-doris/releases/tag/doris-flink-connector-1.13.5-2.12-1.0.0
+https://github.com/apache/incubator-doris/releases/tag/doris-flink-connector-1.12.7-2.12-1.0.0
+https://github.com/apache/incubator-doris/releases/tag/doris-flink-connector-1.11.6-2.12-1.0.0
+
+Staging repo:
+https://repository.apache.org/content/repositories/orgapachedoris-1000
+
+Vote open for at least 72 hours.
+
+[ ] +1
+[ ] +0
+[ ] -1
+```
+
+投票通过后就可以发布到 Maven Central 了
+![](/images/release-stage.png)
diff --git a/extension/flink-doris-connector/pom.xml b/extension/flink-doris-connector/pom.xml
@@ -41,9 +41,9 @@ under the License.
         </license>
     </licenses>
     <scm>
-        <connection>scm:git:git@github.com:apache/incubator-doris.git</connection>
-        <developerConnection>scm:git:git@github.com:apache/incubator-doris.git</developerConnection>
-        <url>scm:git:git@github.com:apache/incubator-doris.git</url>
+        <connection>scm:git:https://git@github.com/apache/incubator-doris.git</connection>
+        <developerConnection>scm:git:https://git@github.com/apache/incubator-doris.git</developerConnection>
+        <url>scm:git:https://git@github.com/apache/incubator-doris.git</url>
         <tag>HEAD</tag>
     </scm>
     <issueManagement>
@@ -76,6 +76,7 @@ under the License.
         <maven-source-plugin.version>3.2.1</maven-source-plugin.version>
         <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
         <doris.thirdparty>${env.DORIS_THIRDPARTY}</doris.thirdparty>
+        <project.scm.id>github</project.scm.id>
     </properties>
 
     <profiles>

diff --git a/extension/spark-doris-connector/pom.xml b/extension/spark-doris-connector/pom.xml
@@ -41,9 +41,9 @@
         </license>
     </licenses>
     <scm>
-        <connection>scm:git:git@github.com:apache/incubator-doris.git</connection>
-        <developerConnection>scm:git:git@github.com:apache/incubator-doris.git</developerConnection>
-        <url>scm:git:git@github.com:apache/incubator-doris.git</url>
+        <connection>scm:git:https://git@github.com/apache/incubator-doris.git</connection>
+        <developerConnection>scm:git:https://git@github.com/apache/incubator-doris.git</developerConnection>
+        <url>scm:git:https://git@github.com/apache/incubator-doris.git</url>
         <tag>HEAD</tag>
     </scm>
     <issueManagement>
@@ -77,6 +77,7 @@
         <maven-source-plugin.version>3.2.1</maven-source-plugin.version>
         <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
         <doris.thirdparty>${env.DORIS_THIRDPARTY}</doris.thirdparty>
+        <project.scm.id>github</project.scm.id>
     </properties>
 
     <profiles>

diff --git a/fe/pom.xml b/fe/pom.xml
@@ -42,9 +42,9 @@ under the License.
         </license>
     </licenses>
     <scm>
-        <connection>scm:git:git@github.com:apache/incubator-doris.git</connection>
-        <developerConnection>scm:git:git@github.com:apache/incubator-doris.git</developerConnection>
-        <url>scm:git:git@github.com:apache/incubator-doris.git</url>
+        <connection>scm:git:https://git@github.com/apache/incubator-doris.git</connection>
+        <developerConnection>scm:git:https://git@github.com/apache/incubator-doris.git</developerConnection>
+        <url>scm:git:https://git@github.com/apache/incubator-doris.git</url>
         <tag>HEAD</tag>
     </scm>
     <issueManagement>
@@ -87,6 +87,7 @@ under the License.
         <jmockit.version>1.49</jmockit.version>
         <log4j2.version>2.17.1</log4j2.version>
         <revision>0.15-SNAPSHOT</revision>
+        <project.scm.id>github</project.scm.id>
     </properties>
 
     <profiles>

diff --git a/fe_plugins/pom.xml b/fe_plugins/pom.xml
@@ -42,9 +42,9 @@ under the License.
         </license>
     </licenses>
     <scm>
-        <connection>scm:git:git@github.com:apache/incubator-doris.git</connection>
-        <developerConnection>scm:git:git@github.com:apache/incubator-doris.git</developerConnection>
-        <url>scm:git:git@github.com:apache/incubator-doris.git</url>
+        <connection>scm:git:https://git@github.com/apache/incubator-doris.git</connection>
+        <developerConnection>scm:git:https://git@github.com/apache/incubator-doris.git</developerConnection>
+        <url>scm:git:https://git@github.com/apache/incubator-doris.git</url>
         <tag>HEAD</tag>
     </scm>
     <issueManagement>
@@ -75,6 +75,7 @@ under the License.
     <properties>
         <log4j2.version>2.17.1</log4j2.version>
         <doris.version>0.15-SNAPSHOT</doris.version>
+        <project.scm.id>github</project.scm.id>
     </properties>
     <profiles>
         <!-- for custom internal repository -->

diff --git a/fs_brokers/apache_hdfs_broker/pom.xml b/fs_brokers/apache_hdfs_broker/pom.xml
@@ -43,9 +43,9 @@ under the License.
         </license>
     </licenses>
     <scm>
-        <connection>scm:git:git@github.com:apache/incubator-doris.git</connection>
-        <developerConnection>scm:git:git@github.com:apache/incubator-doris.git</developerConnection>
-        <url>scm:git:git@github.com:apache/incubator-doris.git</url>
+        <connection>scm:git:https://git@github.com/apache/incubator-doris.git</connection>
+        <developerConnection>scm:git:https://git@github.com/apache/incubator-doris.git</developerConnection>
+        <url>scm:git:https://git@github.com/apache/incubator-doris.git</url>
         <tag>HEAD</tag>
     </scm>
     <issueManagement>
@@ -74,6 +74,7 @@ under the License.
         <maven.compiler.source>1.8</maven.compiler.source>
         <maven.compiler.target>1.8</maven.compiler.target>
         <log4j2.version>2.17.1</log4j2.version>
+        <project.scm.id>github</project.scm.id>
     </properties>
 
     <profiles>
@@ -254,13 +255,6 @@ under the License.
             <version>0.13.0</version>
         </dependency>
 
-        <!-- https://mvnrepository.com/artifact/log4j/log4j -->
-        <dependency>
-            <groupId>log4j</groupId>
-            <artifactId>log4j</artifactId>
-            <version>1.2.17</version>
-        </dependency>
-
         <!-- https://mvnrepository.com/artifact/org.apache.logging.log4j/log4j-api -->
         <dependency>
             <groupId>org.apache.logging.log4j</groupId>