Skip to content

Commit

Permalink
[typo](kerberos)support read jdk auth creds and add some krb tips in …
Browse files Browse the repository at this point in the history
…FAQ (#22535)

support read jdk auth creds and add some krb tips in FAQ
1. about the 'javax.security.auth.useSubjectCredsOnly': https://stackoverflow.com/questions/43660265/java-automatically-uses-kerberos-ticketcache-when-it-shouldnt
2. add tips for `No common protection layer between client and server` and yum jdk version.
  • Loading branch information
wsjz authored and xiaokang committed Aug 11, 2023
1 parent 7f13947 commit 35ad55c
Show file tree
Hide file tree
Showing 4 changed files with 79 additions and 4 deletions.
4 changes: 2 additions & 2 deletions conf/be.conf
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,10 @@ CUR_DATE=`date +%Y%m%d-%H%M%S`

PPROF_TMPDIR="$DORIS_HOME/log/"

JAVA_OPTS="-Xmx1024m -DlogPath=$DORIS_HOME/log/jni.log -Xloggc:$DORIS_HOME/log/be.gc.log.$CUR_DATE -Dsun.java.command=DorisBE -XX:-CriticalJNINatives -DJDBC_MIN_POOL=1 -DJDBC_MAX_POOL=100 -DJDBC_MAX_IDLE_TIME=300000 -DJDBC_MAX_WAIT_TIME=5000"
JAVA_OPTS="-Xmx1024m -DlogPath=$DORIS_HOME/log/jni.log -Xloggc:$DORIS_HOME/log/be.gc.log.$CUR_DATE -Djavax.security.auth.useSubjectCredsOnly=false -Dsun.java.command=DorisBE -XX:-CriticalJNINatives -DJDBC_MIN_POOL=1 -DJDBC_MAX_POOL=100 -DJDBC_MAX_IDLE_TIME=300000 -DJDBC_MAX_WAIT_TIME=5000"

# For jdk 9+, this JAVA_OPTS will be used as default JVM options
JAVA_OPTS_FOR_JDK_9="-Xmx1024m -DlogPath=$DORIS_HOME/log/jni.log -Xlog:gc:$DORIS_HOME/log/be.gc.log.$CUR_DATE -Dsun.java.command=DorisBE -XX:-CriticalJNINatives -DJDBC_MIN_POOL=1 -DJDBC_MAX_POOL=100 -DJDBC_MAX_IDLE_TIME=300000 -DJDBC_MAX_WAIT_TIME=5000"
JAVA_OPTS_FOR_JDK_9="-Xmx1024m -DlogPath=$DORIS_HOME/log/jni.log -Xlog:gc:$DORIS_HOME/log/be.gc.log.$CUR_DATE -Djavax.security.auth.useSubjectCredsOnly=false -Dsun.java.command=DorisBE -XX:-CriticalJNINatives -DJDBC_MIN_POOL=1 -DJDBC_MAX_POOL=100 -DJDBC_MAX_IDLE_TIME=300000 -DJDBC_MAX_WAIT_TIME=5000"

# since 1.2, the JAVA_HOME need to be set to run BE process.
# JAVA_HOME=/path/to/jdk/
Expand Down
4 changes: 2 additions & 2 deletions conf/fe.conf
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,10 @@ CUR_DATE=`date +%Y%m%d-%H%M%S`
# the output dir of stderr and stdout
LOG_DIR = ${DORIS_HOME}/log

JAVA_OPTS="-Xss4m -Xmx8192m -XX:+UseMembar -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7 -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0 -Xloggc:$DORIS_HOME/log/fe.gc.log.$CUR_DATE"
JAVA_OPTS="-Djavax.security.auth.useSubjectCredsOnly=false -Xss4m -Xmx8192m -XX:+UseMembar -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7 -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0 -Xloggc:$DORIS_HOME/log/fe.gc.log.$CUR_DATE"

# For jdk 9+, this JAVA_OPTS will be used as default JVM options
JAVA_OPTS_FOR_JDK_9="-Xss4m -Xmx8192m -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7 -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0 -Xlog:gc*:$DORIS_HOME/log/fe.gc.log.$CUR_DATE:time"
JAVA_OPTS_FOR_JDK_9="-Djavax.security.auth.useSubjectCredsOnly=false -Xss4m -Xmx8192m -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7 -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0 -Xlog:gc*:$DORIS_HOME/log/fe.gc.log.$CUR_DATE:time"

##
## the lowercase properties are read by main program.
Expand Down
40 changes: 40 additions & 0 deletions docs/en/docs/lakehouse/multi-catalog/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,3 +149,43 @@ under the License.
```
14. When using JDBC Catalog to synchronize MySQL data to Doris, the date data synchronization error occurs. It is necessary to check whether the MySQL version corresponds to the MySQL driver package. For example, the driver com.mysql.cj.jdbc.Driver is required for MySQL8 and above.
15. If an error is reported while configuring Kerberos in the catalog: `SIMPLE authentication is not enabled. Available:[TOKEN, KERBEROS]`.
Need to put `core-site.xml` to the `"${DORIS_HOME}/be/conf"` directory.
If an error is reported while accessing HDFS: `No common protection layer between client and server`, check the `hadoop.rpc.protection` on the client and server to make them consistent.
```
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>hadoop.security.authentication</name>
<value>kerberos</value>
</property>
</configuration>
```
16. The solutions when configuring Kerberos in the catalog and encounter an error: `Unable to obtain password from user`.
- The principal used must exist in the klist, use `klist -kt your.keytab` to check.
- Ensure the catalog configuration correct, such as missing the `yarn.resourcemanager.principal`.
- If the preceding checks are correct, the JDK version installed by yum or other package-management utility in the current system maybe have an unsupported encryption algorithm. It is recommended to install JDK by yourself and set `JAVA_HOME` environment variable.
17. If an error is reported while querying the catalog with Kerberos: `GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos Ticket)`.
- Restarting FE and BE can solve the problem in most cases.
- Before the restart all the nodes, can put `-Djavax.security.auth.useSubjectCredsOnly=false` to the `JAVA_OPTS` in `"${DORIS_HOME}/be/conf/be.conf"`, which can obtain credentials through the underlying mechanism, rather than through the application.
- Get more solutions to common JAAS errors from the [JAAS Troubleshooting](https://docs.oracle.com/javase/8/docs/technotes/guides/security/jgss/tutorials/Troubleshooting.html).
18. If an error related to the Hive Metastore is reported while querying the catalog: `Invalid method name`.
Configure the `hive.version`.
```sql
CREATE CATALOG hive PROPERTIES (
'hive.version' = '2.x.x'
);
```
35 changes: 35 additions & 0 deletions docs/zh-CN/docs/lakehouse/multi-catalog/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,3 +149,38 @@ under the License.
```
14. 使用JDBC Catalog将MySQL数据同步到Doris中,日期数据同步错误。需要校验下MySQL的版本是否与MySQL的驱动包是否对应,比如MySQL8以上需要使用驱动com.mysql.cj.jdbc.Driver。
15. 在Catalog中配置Kerberos时,如果报错`SIMPLE authentication is not enabled. Available:[TOKEN, KERBEROS]`,那么需要将`core-site.xml`文件放到`"${DORIS_HOME}/be/conf"`目录下。
如果访问HDFS报错`No common protection layer between client and server`,检查客户端和服务端的`hadoop.rpc.protection`属性,使他们保持一致。
```
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>hadoop.security.authentication</name>
<value>kerberos</value>
</property>
</configuration>
```
16. 在Catalog中配置Kerberos时,报错`Unable to obtain password from user`的解决方法:
- 用到的principal必须在klist中存在,使用`klist -kt your.keytab`检查。
- 检查catalog配置是否正确,比如漏配`yarn.resourcemanager.principal`。
- 若上述检查没问题,则当前系统yum或者其他包管理软件安装的JDK版本存在不支持的加密算法,建议自行安装JDK并设置`JAVA_HOME`环境变量。
17. 查询配置了Kerberos的外表,遇到该报错:`GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos Ticket)`,一般重启FE和BE能够解决该问题。
- 重启所有节点前可在`"${DORIS_HOME}/be/conf/be.conf"`中的JAVA_OPTS参数里配置`-Djavax.security.auth.useSubjectCredsOnly=false`,通过底层机制去获取JAAS credentials信息,而不是应用程序。
- 在[JAAS Troubleshooting](https://docs.oracle.com/javase/8/docs/technotes/guides/security/jgss/tutorials/Troubleshooting.html)中可获取更多常见JAAS报错的解决方法。
18. 使用Catalog查询表数据时发现与Hive Metastore相关的报错:`Invalid method name`,需要设置`hive.version`参数。
```sql
CREATE CATALOG hive PROPERTIES (
'hive.version' = '1.x.x'
);
```

0 comments on commit 35ad55c

Please sign in to comment.